“Data is the new oil.”

Unless you have been living under a rock for the last 15 years, you have probably heard the phrase “data is the new oil.” It was 2006 when Clive Humby, a mathematician and entrepreneur originally coined the phrase – and it has been echoed by thousands ever since. 

Clive Humby probably never expected this phrase to make such a lasting impression but it really was the perfect comparison. Oil and data have a lot in common.

Look around you, in some way or another, oil has played a crucial role in the existence of everything around you. Many everyday items are a product of oil, they are manufactured in a factory powered by oil, then transported by vessels which are also powered by, you guessed it, oil. 

Today, data is making just as much of an impact. Data is used in a multitude of ways that influence decision making in government, public policy, healthcare, science, and of course business.

Society relies on data and oil to make things go. For that reason they are considered valuable commodities – that many governments, policy makers and businesses are desperately trying to get their hands on. Data and oil are power. Without them, making things happen can be challenging. 

Most importantly, when we drill for oil or mine for data, to do so successfully requires some knowledge and the right tools. Otherwise it could be a colossal waste of time and resources. To drill for oil requires knowledge of where the oil might be and how to extract it. It requires machinery, and man power. Once the oil has been extracted, it needs to be refined so it can be used in its full potential. 

The same goes for data. There must be knowledge of where to collect data and how to extract it. It requires specific technology and manpower. Then once the data has been collected, it needs to be refined so it can be used to its full potential. 

Unlike drilling for oil, you won’t break your back trying to collect data. But it does require some effort. Collecting data is the first step in answering those valuable questions before we can refine it and get its true value. 

Here are some best practices for collecting data:

Have a robust internal system:

Collecting data in-house is the best practice when solving internal business challenges or identifying opportunities. To collect data in-house means that information is being recorded and stored internally on a consistent basis. Typically data is collected and stored in a customer relationship manager (CRM). And if your team is using your CRM, you probably have a pretty decent set of data to get insights from. If they aren’t using the CRM to its full potential – you should really consider it. Having some data, even if it is incomplete, is better than having no data at all. 

If you are not sure what to collect, it is still a great practice to capture as much data as you can. It’s never possible to know exactly what’s going to be of value in future. So collecting data in a structured format that can be translated into information and insights down the road and can create new opportunities in the future. And the good news is that data storage is cheap these days, so holding onto that data isn’t going to make much of a dent in your budget. 

Buy data from third parties:  

There are times when a company is unable to collect their own data because they could be a start-up, a company looking to enter a new market, or there is a major gap realized in the current set of data. Instead of starting from scratch, buying data from third parties is a good option. There are plenty of  data sets out there that third party data brokers have carefully taken the time to put together. They have a wide variety of data sets that are usually made up of data that is challenging to collect, data that is challenging to access, or very large sets of data – that could take decades to collect.

If you aren’t sure where to start, Quandl and Explorium are reputable data brokers with plenty of data to go around. 

Use third parties to buy training data: .

If you are looking to partake in the AI and Machine Learning boom and want to take on a project, the algorithms are going to require some training data to learn from. If there is a shortage of training data available, third parties are also a good solution for this as well. Third parties can provide data sets a lot faster and efficiently rather than trying to put together a set of training data

There are many options for training data, check out Hive, Scale, NTT Data, or Weights and Biases.

Use third parties to collect training data:

There are times where third party data brokers don’t have the exact set of data needed to answer that million dollar question. When specific data is needed but maybe your team doesn’t have the manpower, tools, or time to collect data there are third parties that can collect that data for you. Once again, long behold the third party data collectors. One of the most popular examples of this is Amazon Mechanical Turk. A platform that individuals from around the world collect data based on the instructions they have been provided with. 

Scrape data off the internet:

As everyone is aware — there is a lot of data on the world wide web, readily available at the tips of our fingers. This practice does require some expertise to create specific tools. But once that has been handled, very specific data can easily be scraped off the internet, parsed and used accordingly. Please be respectful of the terms and conditions of where you scrape data from though! Only do it if you allowed to do.

For example, something that our team has been successful at while scraping data off the internet for lead generation. Instead of sales reps spending hours prospecting qualified leads, our team created a tool that scrapes data off service providers websites such as lawyers and accountants to generate leads. The information was parsed and then put directly into the CRM to be accessed by the sales team. With the right expertise, a tool can be created to collect data on anything and be used to answer that million dollar question.

Ready, set, mine.

The top two reasons that businesses don’t collect data is because it appears to be either too intimidating or time consuming. What this really comes down to is not seeing value and the potential prosperity that could be uncovered. Data drives decision making and uncovers new opportunities. If data collection isn’t a priority, opportunities and insights are being left undiscovered, while challenges are being left unaddressed – both now and in the future. Which can equate to hundreds of thousands of dollars slipping through your fingers.

Collecting data is the easiest part of the process, especially with a little bit of guidance. Refining it is where things can start to become a bit trickier. Luckily, there are data experts out there who can help you uncover the true value of your data.

Plus ça Change, Plus C’est La Même Chose

Jean-Baptiste Alphonse Karr

In case you don’t speak french (or you forgot almost everything you learned in highschool – looking at you Canadians!). The timeless proverb; “the more things change, the more they stay the same” implies that no matter what turbulent changes we experience, the status quo is reinforced. 

Our first thought jumped to sales operations. Sales ops has come a long way over the last few decades. Before, when many of us thought about sales the first thing that popped into our head was a sleazy, used car salesman, trying to nickel and dime you at every corner. Or the guy in the fedora, going door to door, selling the next best vacuum or tupperware set. Heck, even over the last year we have seen a major shift in sales operations from in-person to almost exclusively meeting our prospects over Zoom. 

Our point is, the sales process has changed a lot over time. And regardless of all the trends and changes in the market, sales operations has always been about selling better, selling faster and selling more. This is a capitalist market-driven economy after all. 

Sales Ops in 2021

It is fair to say that sales has become more difficult over the past few decades. 

Information is more widely available. Competition is stiffer. And for that reason, the modern customer has higher expectations and there must be a sense of trust before a purchase is made. Creating strong honest relationships is crucial to driving sales in the modern day. 

We are also living through the 4th technological revolution. Technology has swept through sales operations and has streamlined many of our day to day processes. Seems like there is a service or a tool for anything you could ever imagine. Then you blink and somehow, somebody has created another tool to make your business even more efficient. 

Despite all the amazing software services and solutions out there, many sales teams are still living in the stone age. Otherwise known as… superstar culture. 

Superstar Culture Sales

Is highly individualistic. It relies on a couple people to carry the team to the sales quota. Energy and resources are poured into the few sales wizards that seem to hold the world on their shoulders. Meanwhile, the rest of the sales team is not being given the support needed to improve their processes.

This culture is bad for two reasons. Firstly, sales superstars are not common. They know their worth, and if they left the company for a better offer, you better pray for another one. 

Secondly, when focus is on the sales wizards, the rest of the sales team is left in the shadows. They are not getting the coaching or mentoring they need to develop into becoming the next up and coming sales wizard. They also know their worth and if work culture is lacking – they will be gone too. And nobody wants a churn and burn culture. 

Data Science Technology Prevents Superstar Culture.

One of the biggest tech changes in recent times (that we would argue is here to stay) is the integration of data science technology into the sales process. This changes decision making entirely. Instead of making decisions based on observation and opinion. We can much more confidently make decisions based on data and numbers. We are not forced to rely on a couple sales superstars to hold things together.

Data science combines domain expertise, programming skills, and knowledge of mathematics and statistics to extract meaningful insights from data. 

By taking a data science approach we are choosing a science-driven culture.

Science-driven culture prioritizes:

1. Technology is used to make our day to day processes more efficient. It is also a crucial tool to collect the necessary data to make informed decisions.
2. Processes are what exactly your team is doing to move opportunities down the sales funnel. By prioritizing processes onus is being put on the functionality of the sales funnel steps rather than the individual. When a problem is identified, we look to how we can fix the process, not the person. 
3. Teamwork is crucial to science driven culture. All processes need to be documented and shared within the organization. When mistakes are made or insights for best practices are discovered they are not to be kept as a secret by the individual. Instead they should be shared to benefit everyone and the organization as a whole. 
4. Skills are acquired when we learn from our mistakes and successes. This is an ongoing process. When we continuously collect data and get feedback we are able to learn and improve in the places we are struggling most. 

Science-driven culture is about continuous feedback, learning and making decisions based on numbers and data. It is about fixing weak processes, not fixing people. 

Why Data Science Works

Data science relies on numbers. Not opinion. Not intuition. But cold, hard, calculable, numbers. 

Metrics must be measurable and objective. If your metrics are subjective, based on observation and personal experience – consider them an opinion. And opinions with all due respect are not important and can’t guarantee results. Without numbers and data your metrics are meaningless and left open to debate. 

Let’s apply the scientific method to the sales funnel using a subjective methodology:

1. Observation: We aren’t hitting our sales quota 🙁
2. Question: Why aren’t we hitting our sales quota?
3. Hypothesis: Hmm, well looks like the sales reps aren’t making enough calls. 
4. Prediction: If the sales reps make more calls we will make more sales. 
5. Tests: *sales reps make more calls*
6. Results: Still not hitting the sales quota.
7. Refine: Well, that didn’t work. Maybe it’s ______. (and repeat).

Using a subjective methodology (aka our intuition), we tend to make an observation, make a hypothesis, and test it.  And rinse and repeat steps 3-7 when your first hypothesis doesn’t turn out to be the problem. Repeating these steps is a massive waste of time and resources. 

However, when we use data science and recognize there is a performance gap somewhere in the funnel. The problem can be easily pinpointed by just looking at the numbers. No need to play the guessing game. From there feedback can be provided and a specific solution can be prescribed that is geared towards that exact problem in the sales funnel. This means a much faster turnaround time when solving issues. Rather than having to repeat steps 3-7 until you are just about ready to give up and try to solve the next problem.

It is beneficial at both an individual and organizational level. From an individualistic standpoint, we can analyze where a performer is successful and where they are struggling. This feedback is personalized and can help the individual to improve where they need it.

On a larger scale, this data can be used by the organization to make a more efficient sales process and provides valuable insights for the next-best action with future customers. 

Got the Data?

Most organizations, once they have reached a certain size, have a formal sales process in place and are collecting data. This ensures that they can continue to scale. A typical sales report breaks down the basic key performance indicators (KPI). Number of calls/emails, conversion rate, opportunities lost, profit margin, customer acquisition costs and so on. Not to say these are bad indicators. They are necessary for every business. The problem is they just don’t tell the whole story. 

KPI’s tend to be high level. They leave out the finer metrics that can give us a more accurate picture of what exactly is going on. It is important to know your industry and know what KPI’s are relevant to the problem you are trying to solve. Furthermore, if you are looking to take a science based approach in sales ops, it is crucial to have somebody who knows how to ask the right questions, can extract the necessary data and come up with meaningful solutions.

Tune in next week,  We’ll be following up with a post on “How to Collect Meaningful Data”

“We are drowning in information and starving for knowledge.” — John Naisbitt.

The Era of Big Data

We have entered the era of big data. For example, there are about 60 trillion web pages; 300 hours of video are uploaded to YouTube every minute, amounting to 10 years of content every day; the genomes of 10000s of people, each of which has a length of 3.8 × 109 base pairs, have been sequenced by various labs; Walmart handles more than 1M transactions per hour and has databases containing more than 2.5 petabytes (2.5 × 1015) of information [1]; We are currently creating around five exabytes a day roughly equivalent to 500 million songs — the amount of data available today is giving machines the possibility to become super-intelligent.

What is ML?

This deluge of data calls for automated methods of data analysis, which is what machine learning provides. In particular, machine learning (ML) is defined as a set of methods that can automatically detect patterns in data, and then use the uncovered patterns to predict future data, or to perform other kinds of decision making under uncertainty (such as planning how to collect more data) [2]. A more technical definition of ML is “a system’s ability to correctly interpret external data, to learn from such data, and to use those learnings to achieve specific goals and tasks through flexible adaptation” [3].

AI vs ML?

It’s worth noticing that often ML is used with artificial intelligence interchangeably. Although artificial intelligence is technically broader than ML and they are different, for the purpose of this note, we treat them the same.

Why Now?

Now, you might ask, how old is this stuff and why is it getting a lot of attention now? Artificial intelligence was founded as an academic discipline in 1956. It because most of ML methods work well when you have a large amount of data (to be precise it really depends on the problem you are trying to solve) and as noted we have entered into an era that there is no shortage of data, yay! Another interesting thing recently happed; our computers got way stronger and faster than before. Analyzing and extracting information from an enormous amount of data is not trivial. It needs a lot of computational power and the introduction of graphics processing units (GPUs), which were developed to speed up the graphics in our everyday computers and especially in the gaming world, to ML world changed the game.

What are Some of the Applications of ML?

What are some of the applications of AI/ML? Good question. Although you might not notice it, ML already is a big part of our lives. Amazon makes its recommendations based on a set of ML algorithms called “Recommender systems” specifically a hybrid of “Collaborative filtering” and “Content-based filtering”. Google uses ML to detect spam emails. Facebook uses ML to show you the content that you would find interesting. Tesla is using AI/ML to make self-driving cars. Intelligence services are using ML to detect criminals in crowds and …

ML and AI are already here. The question is how to align your business to benefit from it rather than getting killed by it.

1. “Cukier K., The Economist, Data, data everywhere: A special report on managing information, 2010, February 25, Retrieved from http://www.economist. com/node/15557443”
2. “Kevin P. Murphy , Machine Learning: A Probabilistic Perspective Hardcover — Aug 24 2012”
3. “Kaplan, A. and M. Haenlein (2019) Siri, Siri in my Hand, who’s the Fairest in the Land? On the Interpretations, Illustrations and Implications of Artificial Intelligence, Business Horizons, 62(1)”.

The current coronavirus pandemic situation has created a sudden change in our everyday lives that was unforeseen just a few months ago. As individuals adjust to the “new normal,” businesses should also take a moment to reassess their situation.

Suddenly, the market is upside down, your team is working remotely, and a good portion of your team probably doesn’t have as much on their plate. Although the market is showing signs of coming back, it might take many months to a few years. This uncertainty presents an opportunity to concentrate on the “fundamental” work that’s always in the “to do” bracket. Tasks that were a lower priority with everyone focused on the daily business operations is now perfect for getting done. It’s time to position your company to leverage the current situation so that your company is super competitive in the “new normal.” Well, let me try to explain:

Utilization = morale

We know that excellent staff morale is key to productivity and utilization. I would argue that those who are most engaged are those that understand the reasons for what they are doing. If they are aligned with the company’s vision and understand their roles and possible impact, they will not need your constant motivation. Nothing creates a downward spiral more than underutilization of staff already suffering from the “Mushroom Syndrome.” Now is the perfect time to create alignment and focus. Typically, teams are too busy when things are going well

I’m guilty of this; you’re guilty of this. Typically, when things are healthy, and the company is doing well, everyone is laser-focused on their commitments and targets. We are guilty of working ‘in the business’ rather than ‘on the business.’

The market has changed, and the world is a different place–get ready for it

Some of the effects of COVID-19 are with us for a long time, and some are likely permanent. Now is the time to get ready for dynamic changes that are new to everyone. Markets after the 2009 crash were different. Similarly, the markets after September 11 also changed. The changes after the two events were different too! What do you need to be ready? Data can tell you a lot about this. It would help if you readjusted some of those fundamental assumptions which worked previously. Do you know which ones they are?

New technology initiatives take time

You have got your teams focused on the next 12 months of work; now you need to start thinking of the future. Once you have set the company’s direction, you need to gather intelligence to determine the criteria for success and failure. To quickly identify where the market is headed and stay on the leading edge. The best way of creating a picture is by analyzing many data. That’s where technology can help identify new trends and emerging patterns, but the intelligence lies in understanding data. Implementing technologies like AI and ML takes time. More than that, gathering data takes even longer, and it requires a lot of trials and error testing. Experiment with it while your business is not at full capacity. Now is the best time for when the market comes back, you won’t have the time.

So what?

OK, you’re convinced that it is a good time to do some fundamental work. Where to start? Start from where the impact is. Try to avoid shiny objects and only do what matters. I’m going to write about this topic next week so please stay tuned! Before that let’s look at some other bullet points.

Measure twice and cut once

You’ve heard the saying “Measure twice and cut once,” well, this also applies to businesses. By that, I mean companies should spend time planning their mid to long-term course before committing the workforce to a particular path. Moreover, that same workforce should be part of the planning process, so they have a stake in the outcome.

During this downtime, it’s your job, as the boss, to create the company vision for the 10–5–1 year horizons. You are responsible for the big picture. Where do you see the company (in terms of revenue, size, product development, market capture, and any other target you have been dreaming about) in 12 months from now, then 5 years, and finally 10 years from now. Identify the targets as accurately as you can and see if your staff think it’s possible. If they like what they see, they’ll provide most of the solutions to achieving the ‘want.’

Try filling in the details of your targets in the following table style:

Does everyone working for you know what the company does? Why it exists? You may think those questions are trivial, but to get alignment across the board, everyone must understand what service(s) the company provides, and why it exists. Now is your chance to consider how the current Covid-19 climate can be an opportunity to beat the competition.

In the following diagram, I created a mind map to identify the business of a company. It needs to be simple enough for anyone to understand.

What YOUR landscape looks like?

Start by identifying your current place in the market and then define how new opportunities are becoming apparent. Let your staff tell you what they think.

Along those lines, I created the following drawing for a company to highlight its position in the market. It shows their competitors, suppliers, partners, customers, and other channels. Again, a mind map is the easiest way for everyone to follow:

More efficient and last longer

Use your best and brightest employees to help set the path forward. Once they fully understand what the company does, why it exists, and where it needs to go, they’re motivated to succeed.

Task your leaders with creating milestones to achieve the goals for the next 12 months, let them demonstrate their preparedness to meet those targets. When that occurs, your job is to shepherd them, provide the resources, and trust them.

Finally, it’s very important to find us on LinkedIn and let us know what you think! Always happy to connect.

We recently chose to have a focus on offering productized Sales Operation consulting solutions. We got our first client and within four months, we reduced their sales team cost by 20% and added $6MM to their projected revenue. Our machine-learning-powered solution delivered more than 4MM sales leads with verified contact information to them and cleaned 97% of the duplicates in their Salesforce CRM. That helped them increase the number of valid sales leads by 400% and increase their 20% YoY growth rate even further. The results were so promising that they already have asked us to provide them with a few more solutions including lead scoring and churn prediction.

It is the right time to focus on investing in technology enablement solutions for Sales and Sales Operations teams. Why and how? Please keep reading and you will find out.

Today, Sales and Sales Operations are where Marketing was a decade ago, ready for disruption by data driven approaches and solutions. Data driven methods and machine learning (ML) have changed the game in marketing in a drastic way; to the extent that one has a narrow chance to get a job in Marketing if at least they don’t have a basic understanding of data analysis. Marketing has become a way more efficient market compared to even 5 years ago. Today, less sophisticated players even with an innovative and on-demand products/services find it extremely hard to deliver their messages to their audience. The last time I checked (August 2018), there were around 8000 companies making technology products to help marketers stay competitive. Sales and Sales Operation are next.

Vast amount of data is gathered by sales teams and other teams every day and it has invaluable amount of information and insights in it. There are numerous inefficient or tedious processes in sales and operations that can be handled by machines. It’s not about replacing humans, it’s about enabling them to do their job and what they are great at more and spend less time and energy on tasks that could be done using machines or in other words by cheaper capitals. It’s about empowering sales teams with insights that are hard for us as humans to obtain but computers can do in a fraction of second.

Now, let me give you a few ideas on how you can hop on this fast-moving train and not be left behind. Here are a few ways you can use data science and ML to take your sales/sales operations to the next level, easily increase your revenue at least by few percent and reduce your costs by another few percent. Not only these solutions improve your business KPIs, they also cause your teams focus on what they really need to do and do what they enjoy the most and become happier in their work.

· Dedupe/Clean Up Your CRM

· Enrich Your Data

· Automate Your Lead Generation

· Score Your Leads

· Cross-Validate Your Leads

· Right Lead for the Right Person

· Reach Them at the Right Time and Place

I’m going to write a detailed article on each of the above and other subjects in the coming weeks so stay tuned! For now, let me give you a short description of each subject.

Dedupe/Clean Up Your CRM

CRMs are the heart of any company. Based on our experience, 10–20% of the records on CRMs are duplicates. That’s 10–20% waste of your budget if you send them a postal card for example adding up to millions and millions of dollars for larger companies. And for your sales reps, it’s 10–20% of their time wasted and constant frustration.

Enrich Your Data

Do you treat a senior person the same way your treat a younger lead? When your reps are cold calling a lead, should they be talking about your solutions being on cloud or emphasize on ROI more? Have your leads been in business for a while or just started? Did they just raise a big round or have been financially struggling for a while?You don’t know if all you have is a name, email or phone number. Nowadays, just by looking up their email, looking at their social media and etc.; we can get you a lot more about someone. Age, gender, music taste, you name it! Knowledge is power.

Automate Your Lead Generation

The leads you bought from source X or paid a person in India to grab off the internet are not that reliable. You need to keep them updated and the only way you can do it is by having access to a resource that can do it for you regularly. However, it’s a repetitive task that shouldn’t cost you every time you do it. You need a smart bot to do it for you on a regular basis.

Score Your Leads

Leads have different qualities and even leads with the same qualities have different chances of conversion. Why should you treat them the same or spend the same amount of time (in other words money!) to convert them? After enriching your lead data and knowing more about them, ML can accurately predict the conversion chance. In the long run, that means millions of dollars increased revenue, happier sales staff and happier you!

Cross-Validate Your Leads

Why would you spend your time and money from leads information that is coming from a single source? We’re no longer in 50s. Everything has a higher speed, people move around and there’s an enormous amount of bad data out there. By cross validating your data with several sources, you are improving your leads quality sometimes by a factor of 10. Guess the result? Millions of dollars increased revenue, happier sales staff and happier you!

Right Lead for the Right Person

Sales reps are human and have different personalities, capabilities and interests. Sally might work better with more senior leads and Alex might have more chemistry with a young lawyer who just opened their practice. By looking at your data, it’s no longer a guessing game or gunshot approach. You give me the leads I have a higher chance of closing. The result? All of the good stuff above and higher predictability of revenue.

Reach Them at the Right Time and Place

Mondays are not great for reaching out to leads, right? Nobody reads their email on weekends, right? Depends! When I was raising capital for my previous company, I found out the best time to contact more senior investors is on weekend evenings. Based on a project, we found out the best time to approach women in a specific demographic is Monday mornings. Email versus phone vs etc.? Depends! The good news is that it’s all hidden in your data you already have. The bad news is that you need to know how to extract it or it’s just sitting there.

Can you afford your competitors benefit from these superpowers without you taking any steps?

Thanks for reading this and looking forward to writing more about this topic in the coming weeks. Sign up for our newsletter, to get our articles straight in your inbox.

We asked one of our clients, a well respected and rapidly growing company in the Vancouver Sales Vertical what impact our AI engine had on their Sales Operations. They told us that within four months we had reduced their sales team cost by 20% while at the same time adding more than $6 million to their projected annual revenue.

We were quite taken aback. Initially we expected a more modest impact as our Machine Learning Solution was relatively new and our algorithms were not fully trained. In fact we had not even begun training our model to predict churn and lead scoring.

This feedback led us to reach out to other firms like Maximizer, FreshBooks, etc and when we asked them about their toughest challenges in their sales and sales operations, we consistently heard the same messages:

• Duplicates in their CRM systems were causing them frustration and the lack of verified contact information was wasting valuable prospecting time.
• There was unhappiness with having to scale up a sales team when it was possible to enable already trained staff to work more effectively.
• Many leaders mentioned to us that although there was a surplus of AI solutions for marketers, they were disappointed that so few effective machine based learning solutions existed for sales and sales operations.
• Others commented that they were frustrated at managing vast troves of data knowing that they were missing out on key insights and having to spend way too much time managing their ever growing data piles.

Over the next few weeks we will explore deeper into these topics. Sign up for our newsletter, as we share our findings around:

• What parole applications can teach us about timing in sales.
• How to calculate the real costs of CRM duplicates
• How AI can enrich your data beyond basic demographic data
• Which smart-bots work best with Machine Learning to optimize your lead generation
• Why a 5% increase in your lead scoring is the best thing to accomplish this week
• How correctly matching a lead to the right salesperson makes for happier customers and a more engaged sales force.

Stay tuned!

We recently helped a client to clean up their Salesforce CRM. First, we made our proprietary solution to find all of the duplicate records that would pass the client’s criteria and then had to use Salesforce native merge functionality to remove the duplicates. By doing the deduplication process, we helped them save their Sales Reps time by an estimated 10% to 20%.

After a few hours of searching online, we figured that there is not much documentation on how to interact with the Salesforce API we had to work with, their SOAP API and particularly no one had talked about using their merge functionality. This article is written to make it easier for other developers that are planning to do what we did and save them time, a lot of it. Let’s say Salesforce API and SOAP APIs are not the most fascinating APIs to work with!

Our Programming Language:

You can send a SOAP request through any programming language that you like. Here, we are using python as it is becoming more and more popular especially among data scientists and other data related experts. Extending it to your favourite language should be quite straightforward though.

A Bit of Background:

Let’s look at a bit of background on SOAP protocol that can be found on Wikipedia:

“ SOAP (abbreviation for Simple Object Access Protocol) is a messaging protocol specification for exchanging structured information in the implementation of web services in computer networks. Its purpose is to provide extensibility, neutrality and independence. It uses XML Information Set for its message format, and relies on application layer protocols, most often Hypertext Transfer Protocol (HTTP) or Simple Mail Transfer Protocol(SMTP), for message negotiation and transmission.”

So essentially, we need to send an xml message to Salesforce API that is based on a given standard. You need to look at the Web Service Description Language of Salesforce (WSDL) to (hopefully J) find out how to format your xml message. The good news that we’ve done this for you for merging.

Let’s Get Started

First, we need to make a nice header to send with our API call. Here is how the header should look like:

headers = { 'Content-Type': 'text/xml', 'Accept': 'application/soap+xml,application/dime, multipart/related, text/*', 'Authorization': 'Bearer ' + self.session_id, 'SOAPAction': 'merge', 'Sforce-Auto-Assign': 'false', 'charset':'UTF-8'}

It should be quite self-explanatory; if you are wondering about ‘Sforce-Auto-Assign’: ‘false’ part, it’s helping us to avoid any weird default assignment rule that your Salesforce might have.

How about the body?

Now it’s time to actually make a nice xml message. We are doing a bulk merge request and hence we have a list of records that we want to merge. We have stored that list into a list called data and each row of data is a list containing the following in order:

• sf_type: type of the records to be merged (Account, Contact, Lead)
• master_id: Salesforce Id of the master
• victim_id: Salesforce Id of the victim
• valuesToKeep: Please see below

For example, the first row of data can be:

['Lead', '00Q2G00001atpfx', '00Q2G01001atqfl', {'Industry': 'Law', 'Phone': '+1 111 111 1111'}]

One very important note. Salesforce does not keep the info from the victim even if the fields for the master are empty! You need to tell it explicitly to overwrite those fields. As a result, we made a dictionary ( valuesToKeep) of field-names (keys of valuesToKeep) and victim values (values of valuesToKeep) we want to overwrite into the master record. For example, in the above, we are asking Salesforce to use ‘ Law’ as the final record’s ‘ Industry’ and ‘+1 111 111 1111’ as ‘ Phone’.

Finally, let’s put all of these together and form xml body:

body = '''<?xml version="1.0" encoding="utf-8" ?> <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:urn="urn:enterprise.soap.sforce.com" xmlns:urn1="urn:sobject.enterprise.soap.sforce.com" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><soapenv:Header> <urn:SessionHeader><urn:sessionId>{sessionId}</urn:sessionId></urn:SessionHeader> </soapenv:Header><soapenv:Body> <ns1:merge xmlns:ns1='urn:partner.soap.sforce.com'>'''.format(sessionId = self.session_id) for rec in data: sf_type, master_id,victim_id,valuesToKeep = rec body = body + '''<ns1:merge><ns1:masterRecord> <ens:type xmlns:ens='urn:sobject.partner.soap.sforce.com'>{sf_type}</ens:type> <ens:Id xmlns:ens='urn:sobject.partner.soap.sforce.com'>{master_id} </ens:Id>'''.format(sf_type = sf_type, master_id = master_id) for rec in data:
    sf_type, master_id,victim_id,valuesToKeep = rec
    body = body + '''<ns1:merge><ns1:masterRecord>    
    <ens:type xmlns:ens=
    'urn:sobject.partner.soap.sforce.com'{sf_type}</ens:type>    
    <ens:Id xmlns:ens='urn:sobject.partner.soap.sforce.com'
    {master_id}
    </ens:Id>'''.format(sf_type = sf_type, master_id = master_id)    for item in valuesToKeep.keys():
        try:
            value = valuesToKeep[item]
            body = body + '''<ens:{item}   
            xmlns:ens='urn:sobject.partner.soap.sforce.com'>
            {value}</ens:{item}>'''.format(item=item, value=value)        except:
            passbody = body + '''</ns1:masterRecord><ns1:recordToMergeIds>{victim_id}</ns1:recordToMergeIds> </ns1:merge>'''.format(victim_id = victim_id) body = body + '''</ns1:merge></soapenv:Body></soapenv:Envelope>'''

Let’s send our request!

We use the library requests to the API call. You are free to use another method that suits you.

import requests 
from simple_salesforce import Salesforcesession = requests.Session() 
sf = Salesforce(username = username, password = password, security_token = token, session = session, domain = domain)sf.session.request(method = 'POST', url = 'https://' + sf.sf_instance +'/services/Soap/u/' + sf.sf_version, data = body, headers = headers)

You need to provide for username, password, and security_token. domain is ‘test’ for Sandbox and None for the other cases.

Note: Salesforce API version 38 is problematic. You want to avoid it.

If everything goes well, Salesforce returns you a xml that contains no error message. Otherwise, it will try to help you understand where it went wrong, and you can correct any potential mistakes.

I wish you good luck and enjoy working with Salesforce API! Feel free to contact me if you had any questions.

The first time I felt the gap was during my PhD years when one of my close friends was looking for a job. He is a smart guy with great talent and a PhD in computational physics from a top university, UBC. He connected with many great companies whom really needed his expertise and had 5 successful interviews with each of them. However, they never hired him, as they thought he did not have enough job experience! They preferred to hire a candidate with a BSc with a couple of years of work experience than a recently graduated PhD. He started a tech company instead and after a year and half, had a successful exit.

There is a massive disconnect between leadership teams of companies who mostly come from non-technical backgrounds and the technical world, let alone academia. The gap is quite dangerous for the existence of these companies. We live in an era where technology plays a crucial role in companies’ success or ultimate failure. The leaders of a company need to have accurate understandings of the transition of the technologies used in their sectors and other sectors. They need to understand how they can benefit from the new technologies available and how they should plan for the future of their business, given that what used to be science fiction would be a reality soon.

In this post, the emphasis is on artificial intelligence (AI), machine learning, and data science and their potential contribution to a company’s success or failure. Until now the main beneficiary of AI has been the technology sector. However, more non-tech companies in a broad range of industries are starting to worry that AI could scorch or even incinerate them. In 2017, firms worldwide spent around $21.8 billion on mergers and acquisitions related to AI. Around 85% of companies think AI will offer a competitive advantage, but only 5% are “extensively” employing it today. This provides plenty of opportunities for the first-movers and substantial threat to others. McKinsey predicts that digital innovations including AI will provide an estimated economic impact of about $370 billion per year worldwide in 2025, so how would you benefit from this exponential opportunity?

Based on my personal experience, most business leaders have limited knowledge about the potential of AI for their business. Worse, their companies do not have experts who can provide consultation. Even the companies who have teams of data scientists and/or machine learning scientists cannot benefit from the knowledge of their team as the two sides live in very different worlds. Technical teams are mostly concerned about the details of projects and do not focus enough on the main concerns of the business team and the OKRs.

Now what can you do to make AI work for your business? McKinsey has a good report on this. I definitely suggest reading it. Here are some of my additional recommendations:

• Get an AI Advisor: Having advisors who are experts in a given field is a great way of making the right decisions. It is a common practice for startups to have advisors as they don’t have the resources to hire enough senior talent, but I believe it is a fantastic way for any company to stay competitive and informed about the new opportunities. There are many advisors who have strong technical backgrounds, but as a result of their business experience have learned the language of business teams. They can communicate effectively with the C-level team and even play the role of a bridge between the technical teams and the business teams.
• Attend/Organize Conferences/Meetups: No matter how experienced you are at your business, everyone can learn and it is really easy to learn new stuff in a more technical conference. Conferences gather a wide range of expertise with different ongoing projects. The speakers spend hours preparing for talks and the topics of these talks can give you a good sense of the technology trends in your industry and the other sectors. Meetups gather a more diverse set of attendees with different goals for attending the events. In addition to the usual networking opportunities and getting your brand out there, more technical meetups attract “nerdier” attendees, resulting in interesting conversations.
• Find an Expert Partner: You do not need to do it on your own. With so much interest in AI, it can still be really hard to attract good talent in that domain. As the demand for more hands-on consulting projects is increasing, more consulting firms are being born, which are focused on developing customized AI solutions. Think of them as an advisor that helps you to build the solutions in contrast to what McKinsey says. As not every company needs an IT department, not every company needs a team of AI/machine learning scientists.

Whatever you do, don’t sit still and start your AI transformation today.