Retail analytics guide
Within the span of a few years, the field of retail has gone through remarkable changes: data analytics has become an essential support line for retailers in the struggle of staying competitive in an ever-changing, fast-growing industry.
Since you’re reading this, you are probably at a point where you’ve realised the importance of capitalising on data in retail. You’re likely here because you’re either trying to figure out where to start or how to further your data analytics journey.
At the entry point of building a data culture most retailers will use some kind of Business Intelligence tool, such as Tableau or Power BI. These software offer a “boxed solution” to general needs.
A “one-size-fits-all” service for unspecified issues that every retailer has: they make reporting and visualizing much easier and they can also be used for basic data analysis.
However, these boxed solutions aren’t flexible enough to handle large scale data projects that aim to tackle industry and business specific goals. An analytical tool on its own, without data experts with industry specific experience, should not be expected to create a vital data culture.
For that i.e., to be able to carry out elaborate data analyses, advanced data analytics projects should be launched. Furthermore, an even better approach is to use the help of professionals.
Until then, let us give you a compass, one you can always look at whenever you need guidance on your data journey.
Read on to find out:
How to choose your next project
- Churn prediction
- Campaign Affinity
- Direct Mail Response Statistics
- CRM Campaign Insights
- CRM Recommendation
- Seasonal pricing
- Known Value Item Identification
- Promotional Pricing
- Dynamic Pricing
- Long-term pricing strategy
How to choose your next project
Advanced data analytics is essential for continuous, long-term development – perhaps even more so in retail. However, it is not always clear where to start, which potential direction to take when there isn’t a specific problem that needs a solution.
One thing is for sure though: you will definitely need a data collection plan and a well-structured way of maintaining and processing said data.
An easy and straightforward way of collecting useful data from your customers are loyalty programs (mentioned in the Promotional pricing and Dynamic pricing use cases below). Loyalty programs are an easy go-to solution to understand (some of) your customers better. However – just with any data collection –, how you handle that data is a crucial question. Make sure your legal / compliance and data teams are up to date on regulations and data is collected and stored appropriately.
That is also amongst the reasons why in many cases it often makes more sense to work (at least partially) together with a professional data analytics team.
This graph we put together will come in handy in those exact situations: we’ve visualised different (advanced) data analytics projects by complexity (Y axis) and business impact (X axis).
By complexity, we mean the necessary resources for completion as well as the changes needed in process management and the algorithmic changes; while business impact refers to the mid to long term impact a project has on your business metrics.
These use cases may be pursued with your BI team as a pilot data project in-house or with the help of a professional data consulting team.
It’s important to keep in mind that “hiring” (i.e. working together with) a consulting company doesn’t mean that your internal team is completely out of the picture: a complex project requires large scale involvement from your team’s side as well. At the same time a team in place is not a prerequisite of working with external professionals.
Or in simple terms: you don’t need a team, yet if you have one, consultants can and likely will work in cooperation with them.
Therefore, whether you choose to solve everything internally or decide to work with an external data science team, if you’ve just started practising data-driven decision making, it may be wise to choose a new project from the left lower quadrant, since you don’t want to bite off more than you can chew.
In practice though, when your team is faced with a business problem that has to be resolved as soon as possible, you may just end up with a first analytics project that is of great complexity.
In that case, there is no need to panic as you can always find makeshift solutions. What you should keep in mind though is that a temporary solution is just that. It may provide a fix that works for a while, but if you are planning on long term development with sustainable results, sooner than later you will have to invest resources in improving your answers to data related business questions.
For that matter, let’s look at some of the main problem areas that we too often see arising with our retailer clients. A bit further down we then talk in detail about the use cases shown on the chart above, classified under the problem areas we’ve selected.
Main problem areas in retail
There is a large variety of challenges in retail, even when we just consider the ones that can be supported by advanced data analytics. These can be separated into the following domains:
- CRM
- Sales boosting
- Pricing
- Logistics and Operations
- E-Commerce
Each of these domains contain challenges both in the low and high ends of the spectrum of complexity. However, it may not always be clear which use case fits a particular problem the best.
We’ve collected a few ideas from each of the main problem areas that could be used as a solution to the most common challenges you’ll face in retail.
To simplify the compass for you, we have structured the uses cases around a couple viewpoints, as follows:
- what - what your end goal may look like
- when - among what circumstances you might utilise it
- why - why it makes sense to allocate resources on this
- how - to get started with / implement the particular use case
Let’s dive in!
CRM
The point of a well-maintained CRM tool goes way beyond managing your leads. A CRM that receives structured data from the right metric points (i.e. data that is consistent with the same metrics defined at all entry points) is a gold mine that should be systematically utilised in your marketing and sales activities.
With actionable, insightful data in your CRM you can understand your customers and their behavioural patterns better, which then leads to more efficient marketing operations and better user experiences.
Churn prediction
What: |
Focus on customer retention by identifying potential churners well before they actually churn. Predict which customers are most likely to leave your business based on their buying patterns and prevent them from leaving by offering personalised solutions to their pain points. |
When: |
You should always actively try to prevent your customers from leaving, but it is a lot easier if you have a loyalty program, since this way, you can link transactions to customers. After first launching a loyalty program, retailers usually focus on customer acquisition and loyalty building, while this use case usually comes into play later on. That being said, it’s never too early to start planning your customer retention strategy – and churn prediction is an important part of that. Therefore, if you are operating a loyalty program, you should always utilise its advantages – in this case, by monitoring your customers’ buying patterns and intervening when a customer is drifting away. |
Why: |
Acquiring new customers can cost a lot – it’s usually way more expensive than retaining existing ones. Not only is it hard to get a new customer to come and shop at your store, but it’s also difficult and pricey to build their loyalty – not to mention the fact that your loyal customers usually spend more than new clients. Preventing your existing customers from leaving is one of the most vital parts of any CRM strategy. |
How: | The first step should be to create a churn definition if you do not have a commonly used one. This, of course, should be backed by exploratory data analysis. Finding the most fitting churn definition is the most crucial part of this use case, as it will make or break your project: you can hardly ever tell if a customer has truly churned, so it all depends on the definition you use. After deciding on a feasible definition, you can start planning and building a model using features that contribute to the prediction of churn the most. Keep in mind that these models are never a hundred percent accurate: it’s nearly impossible to predict your customers’ next move with maximum accuracy. Lastly, you should build a campaign strategy for customers in the danger zone and monitor the effects of your marketing actions on your churn rate. It is very important to run your campaigns regularly: a few marketing actions scattered randomly throughout the year is not going to stop your customers from leaving. If you’re interested in this use case, read this summary on one of our previous projects that focused on churn prediction to find out more about this topic! |
Campaign Affinity
What: |
Increase the effectiveness of your marketing actions by measuring your customers’ willingness to participate in different types of campaigns. Measure success by analysing the efficacy of your messaging, channel selection, timing, and angle, then target customers according to their affinity scores (i.e., what messages/channels etc. they will likely resonate with). |
When: |
Campaign affinity should be measured when marketing campaigns are conducted regularly. Whether you are creating a research-, a promotional-, or a retargeting campaign, different customers will react to different approaches, messaging angles, offerings etc. on different channels. If you are not looking to waste your resources or – what’s even worse – to annoy your customers with irrelevant outreaches, you should always try and target the right customers with the right campaign on the right channel. |
Why: |
When you’re more precisely able to target customers that you know are likely to respond to particular types of marketing actions, it makes your campaigns much more efficient. It can not only increase campaign success rates and lower Customer Acquisition Cost (CAC), but it also means you don’t spam customers with offers that are irrelevant to them. Fine-tuning your campaigns will not only benefit your short-term goals of better response rates but will also have an effect on your customers’ long-term satisfaction with your brand. |
How: |
In this case, you need a decent amount of historical data about past campaigns, and customers who responded to each action. For accurate targeting and higher success rates (with your messaging and marketing in general) you should create custom thresholds based on individual customer campaign affinity scores (generated by campaign type/channel/goal). Then you can assign a response probability to each customer based on their previous (historical) behaviour patterns, and you can recalculate this probability after every future campaign. |
Direct Mail Response Statistics
What: |
Improve your brand image, conversion rates and customer satisfaction by making sure your direct mail campaigns are not just a waste of mail storage. Whether you are building a relationship with new customers, creating buzz around news about your business (e.g.: an event, product / services feature, expansion etc.) or simply staying on top of mind of your customers, email marketing still is a great option. That is if you are doing it right. For that you need to be able to create relevant, informative (to the point) and preferably personalised email flows. Thus, once again, you have to be able to understand your customers on a deep level and know what certain customer segments resonate with best. |
When: |
As mentioned above in the “What” section, there are a variety of scenarios in which direct mail is more than a viable choice. Whatever the email's topic or secondary goal is (as the primary goal is likely to engage your prospects or customers), you want to make sure that you are sending the right message to the right people, possibly at the right time. Thus, whenever you’re utilising direct mail in your funnel (Top Of Funnel or TOFU, Middle Of Funnel or MOFU, Bottom Of Funnel or BOFU), it is essential to set up the right metrics and with that keep track of the performance of your email campaigns. |
Why: |
Just as with any form of content creation and marketing operation, you are making your own job much harder and way less effective if you are not tracking actionable metrics. Especially with outbound activities, you want to make sure you are not being counterproductive. Putting a lot of effort into outbound campaigns only to see no effect or worse, an increase in dropout and churn rates is an avoidable outcome. Finding the right amount of outbound communication is hard: you don’t want to flood your customers with emails that they won’t even open and/or will only generate frustration for them... but you also want to stay top-of-mind and to be able to get your offers and news to them. Being able to compare and track your campaigns’ performance from the timing through the tone and length to the subject lines is key to a highly efficient DM campaign strategy. |
How: |
First, you need to identify and standardise your KPIs – you need to measure the same indicators if you want your actions to be comparable. You also want to make sure that you are not fooling yourself with vanity metrics. Measuring things that look pretty but are in no way actionable will not help your efforts. For example, simply looking at subscription rates will not help if your open rates are negligible, while open rates don’t mean much without the Click Through Rate (i.e., how many ppl actually clicked on the Call To Action in your mail). For effective -direct mail- campaigns you should refer back to your ideal customer personas and (if you have data already) on customer segment behaviours, so that you can utilise those insights for better personalisation with your customer groups. Once you have your metrics set up, you can conduct A/B, multivariate, or sequential testing or other methods to understand what does and what doesn’t work for you and your customers. |
CRM Campaign Insights
What: |
Keeping track of the results of your activities is a standard in all data use cases (and also in any business operation). Monitoring how your campaigns perform helps you understand what type of marketing activities engage customers the most. Therefore, choosing the best KPIs (i.e. on what basis are you determining success) for campaign efficiency is crucial. |
When: |
This should be a given if you’re running CRM campaigns regularly. Just like in the case of campaign affinity (where you measure each customer’s willingness to react to a given campaign, which you can read more about in the Campaign Affinity use case description), you want to set up the same metrics for each of your campaigns and track them regularly. And by regularly we mean tracking each and every campaign to be able to improve your efforts. |
Why: |
Having your campaign evaluation methods standardised simplifies your tasks and speeds up the process as well. Moreover, without standardised evaluation you are shooting in the dark. Spending time, human resources and money on campaigns that you can’t accurately analyse afterwards and can’t compare to your other efforts is a Sisyphean task (i.e., it’s a lot of futile effort). |
How: |
Identify KPIs that should be measured for each campaign so you will be able to compare their performances. When you are setting up these KPIs, make sure to choose meaningful, actionable insights. Do not fall into the trap of measuring a vanity metric that looks nice but has little to no actual added value to what it is that works and what you might have to improve. When planning a campaign, you should consider using control groups since their presence allows you to confirm your results. Be careful though: your control groups should be selected on the basis of comparability of the target group, and when selected incorrectly, control groups can give you false results. You should also monitor your campaigns’ long-term effect by keeping an eye on the behavioural differences of your target and control groups over the long run. Some campaigns have a short-term effect, but some will have an impact on your sales even weeks after you ran them. Once you have your KPIs then you want to standardise your campaign evaluation processes. You can apply this to past campaigns as well. |
CRM Recommendation
What: | Increase campaign efficiency and Customer Lifetime Value (CLV) and boost campaign turnover by utilising personalised marketing campaigns and targeted recommendations. Improve customer experience by providing accurate product (/service) recommendations to your customers based on their buying habits, behaviour patterns, customer service feedback and the choices (and feedback) of customers with similar patterns. You can read about a similar use case for your webshop here. |
When: |
Once you have a steady (preferably structured) flow of data that you can maintain, you should start analysing your customers’ buying habits, behaviour patterns and feedback. Then, with that insight, you can take your marketing strategy to the next level, by introducing personalised marketing actions and campaigns. |
Why: |
Improving your customers’ experience by providing them with custom recommendations will not only potentially increase basket size and shopping frequency. It will also improve customers’ perception of your business by providing fitting choices. By doing so you will increase their loyalty and the likelihood of them recommending your business to others. In short: by introducing personalised product/service recommendations, you can boost your customers’ loyalty, increase their spending, and attract new customers. |
How: |
Send your customers relevant offers: products that other customers with similar behavioural patterns purchased, product pairings, or coupons for certain products. You can also prevent customers from spending their money elsewhere if you identify products they are likely to buy at other stores based on other customers with similar behavioural patterns. By understanding what products they are most likely to buy, you can adjust your offers and increase their basket sizes. You can basically make them an offer they (can’t) won’t refuse. You can also reward customers with a discount on their favourite products to increase their loyalty – and to get them to come into the store (or on to your online platform) more often than usual. Collect relevant data from your customers, get to know their habits and send them relevant offers (products that other customers with similar behavioural patterns purchased, product pairings, coupons for certain products etc). |
Sales Boosting
Getting a lead to convert is great, although it should not be your end goal. Getting a lead to become a long-term customer who refers your business to other leads is what you should be aiming for. For that to happen your customers’ journey should not end with a single sale. That is why CRM use cases are of key importance and also why you should test different sales boosting use cases.
With the tools at hand today, the abundance of insightful data available on your customer base, sales boosting is no longer a question of how but a question of when.
Cross-sell
What: |
Boost revenue by finding item pairs that usually sell together. An important distinction in this particular use case is that you should be looking for items that are connected in one way or another and not simply bought together in some odd cases as part of a big shopping – e.g., people buy groceries and beauty/bathroom products occasionally on the same day, yet suggesting a shaving cream with a bakery product would not make much sense |
When: |
If you’re not collecting customer data (i.e., you don’t have a structured data plan and a clear goal, and you don’t have strategically set up metrics and data collection points), or you are looking for a low hanging fruit for your first data project (which you most certainly should), this is one of your safest bets. You can boost your sales by analysing transactions and finding products (/services) that are frequently purchased together. |
Why: |
It is a quite straightforward process with not as many variables as some other, more complex use cases. Finding products that sell well together can provide you with an easy way of boosting your sales through promotions. Customers are not always intent based buyers. That is, a customer may arrive at the checkout with a product (/service) that they did not intentionally look for at the start of their journey. Moreover, a customer may be intentionally looking for a certain product but will have little knowledge of the need for (or comfort of) pairing it with a different product (e.g., a new iPhone buyer may not know that they need to buy an adapter for their charging cables as it is no longer included in the box). Thus, recommended items (or the process of cross-selling) will not only benefit your sales but it will likely make your business appear more professional -- or at least create a great user experience. |
How: |
First and foremost: make sure you are collecting useful, actionable data from your customers, create a plan with your data collection and set up the right metrics. Once you have all that, analyse your transactional data and find products that are likely to be purchased together, then conduct promotions accordingly. Make sure to also analyse your promotions (as mentioned above) constantly so you know how well -in this case- your pairings were predicted. Spikes in sales can be caused by a large variety of reasons (some you have little to no power over), thus creating promotions and direct campaigns only make sense if you are actually recording and following up on data throughout them. That way you can – more precisely – know whether your promotions were effective or there are some tweaks to be done. About that: it’s ok not to get your campaigns right on the first go (or to have some not work on the 542nd go). Build, measure, learn. |
Segmentation (customer, store, product)
What: |
Identify subgroups among your customers, stores, and assortment based on common characteristics and patterns. Creating segments is a data use case that can affect almost all above-mentioned retail data challenges. That is why we have mentioned the benefits of understanding your customers better and forming groups based on their behaviours and characteristics continuously in other use cases. However, besides segmenting customers, you can also segment your stores and products. This way, you don’t have to make business decisions separately for each and every one of your stores: handling them in clusters will make everything easier and more efficient. |
When: |
This is a use case that should be conducted on a systematic basis. The basic requirement of course is an adequate amount of historical data so clustering based on relevant aspects (e.g., your customers’ buying history) makes sense. Segments should be revisited and tweaked over time, and segmentation should be done for each of the regions (areas with significant differences in audiences for e.g., regions / countries) you operate in. In the case of customer segmentation, the idea is to understand what drives the decisions of certain customers, and thus how groups can be formed around those decision-making patterns. By doing so, you’ll be able to create more personalised offers and campaigns to your customers. |
Why: |
When done right (i.e., based on actionable insights – demographic segmentation is useless on it’s on own, unless it is paired with behaviour patterns, purchase data etc.), segmentation can help you find the right ways to engage with and activate different groups of customers. Dividing customers into groups based on similarities is an essential step towards personalization. However, by segmenting your customers, you also segment products and even stores implicitly. Thus, segmentation (as mentioned above) can affect most other use cases. A well-established segmentation can be used in pricing and assortment optimization, demand forecasting, DM and other campaign tweaking etc. |
How: |
Analyse your customers’ behaviour (such as shopping pattern, favourite product groups, average spendings, etc.) and demographic data, and find groups of customers that are similar. See what the most active, highest spending, most regularly visiting etc. customers have in common, and analyse that data so you may be able to move customers from less active, less prone to spending to these groups. There are many different definitions and features that you can use to create customer segments, and choosing the best fit can be tricky, as it all depends on what you want to use segmentation for. For example, if your goal is to upgrade your customers from being an occasional visitor to a loyal one, you should consider RFM (Recency-Frequency-Monetary value) segmentation, which is one of the basics. However, if you’d like to find out which product category a customer is most likely to buy, you could choose shopping mission segmentation. In this use case, the objective is to identify the goal which usually drives a particular customer during their shopping journey. You can then use this information to send your customers relevant offers. Price sensitivity can also be a separating factor in customer segmentation, but also can be used for stores as well: we can segment stores based on the ratio number of their price sensitive and not price sensitive customers. This can be useful in assortment planning and pricing. All of these techniques can be used only if we have access to loyalty customer data. But what happens, if such data is not collected by a retailer? Market basket segmentation for example is not a customer but rather a store segmentation technique which does not require loyalty data. This type of segmentation is based on the cross-buying of products: we are trying to identify item pairs that have high association scores. This can be used in in-store layout and promotion configuration in different stores. |
Pricing
Certainly, it is no secret for you, but your pricing strategy should not only consist of a well-balanced margin for your products.
Again, with the development of analytical tools you can easily map your existing and potential customers’ behaviour patterns and adjust your product (/service) pricing to fit a wider range of clients.
This in no sense means that you should always aim to address more customers as finding the ones most interested in your solutions will always be more profitable. At the same time, with the right pricing strategies, you can always engage more people.
Seasonal pricing
What: |
Increase turnover on your product range by implementing seasonal pricing in your pricing model. |
When: |
If you are operating with a wide enough range of assortment, it is more than likely that certain products will sell better in certain seasons: e.g., summer season, holiday season, long weekends, music festivals, local events etc. If you’ve observed that during certain time periods the sales of some of your product range increases, while others cause them to drop, you should use that finding to your own benefit: optimise your prices. It’s good to keep in mind though, that the prerequisite for a proper seasonal pricing model is a data based long-term pricing strategy. |
Why: |
It goes without saying that certain changes in the weather bring with them changes in customer needs and thus changes in the performance of some products. Besides weather related events, regional and local events can also affect customer behaviour: some products may sell better during certain local or regional events (like alcohol during a local sporting event, or a major one such as the Super Bowl). Understanding the cultural characteristics of each region where you operate stores in and adjusting your prices to seasonal attributes will increase your profit. |
How: |
First, you should identify seasons and events that influence your sales in a certain region, then you analyse your products’ performance changes during them. Once you have the data, you should pick out products that are affected by these seasonalities (ones whose purchasing performance changes seasonally). The key points to keep in mind are: 1. which offerings to change; 2. what’s the appropriate time to apply a seasonal price for said offering; 3. what is the right way to apply said seasonal change. Once you have these, since traditional pricing methods won’t work here (due to the specificity of season and product pairs) you will have to experiment a bit (A/B, multivariate testing etc.): set the price a bit higher, a bit lower, and see what happens. Make sure to clean your data from other external and internal effects to balance out disturbances. Also make sure to keep your testing from a few days (for e.g., in the case of Easter) to around 2 weeks as seasons are short, you’ll need your data fast (also keep tests to a maximum of 4 weeks in general). Naturally the necessary length of your testing will correlate with the intensity of traffic on the selected product, which makes defining the exact time frame dependent on the product. After a few tests, you should be able to pick the right price for the right season. |
Known Value Item Identification
What: |
Optimise your general product pricing by tracking and identifying products that drive your customers' price-image the most – Known Value Items (KVI) are items that outstandingly affect the price perception of your customers. |
When: |
Almost every retail setup has their own KVIs, but not everyone takes the time and effort to identify and use them the right way. This method has been used as a component of the pricing strategy of retailers for a long time, however, it is rarely combined with today’s technology. If you have a wide range of assortment and would like to price your products in a way you both improve your sales and engage your customers, you should consider incorporating data based KVI identification in your pricing strategy. |
Why: |
Customers – more often than not – will shop at retailers that offer the best price of frequently purchased products. If you know which products are your known value items, you can not only improve your profitability, but also improve your customers’ shopping experience – which is a must if you want to be known as the first choice for consumers. |
How: |
First, we should mention that you can identify known value items on different levels of your product hierarchy, and you can also find known value categories – product categories that drive price perception the most and include a higher number of known value items. To identify your known value items, you should first analyse your data such as sales, purchase frequency, customer involvement, etc. You can do so on any level of your product category: on a retailer level (in case of a grocery store), you may find that among others, milk, bread, eggs, butter are known value items. On a category level, however, certain brands of beers would be your known value items among alcoholic beverages for example. After identifying your known value items, you should build a pricing strategy in view of your competitors. It’s important to add that different locations may have different KVIs, so if your stores are scattered around in areas that are dissimilar in purchase power, average age and other social/behavioural aspects, you should cluster your stores first, and handle these clusters separately. Next, you can analyse which products are usually purchased together with KVIs and use price bundling or product bundling, or simply cross-merchandising as ways to increase basket sizes. |
Promotional Pricing
What: |
Generate quick demand and / or boost loyalty by reducing the price of promotional items temporarily or in case of multiple items bundled together / bought systematically (loyalty deal). The most prominent examples of promotional pricing can be utilised with different goals in mind, in vastly different ways. You will have to consider what are the ones that fit your budget and are most applicable to your operations. Some of these are:
In most cases, these strategies are run with a limited time frame or limited stock availability and are supported by marketing activities. |
When: |
If you’ve run promotions before or plan promotional activities regularly, but it seems like your promo pricing actions don’t have a significant effect on your sales performance, you should rethink your whole promotional pricing strategy, and rely on data analytics more. The question of when heavily depends on the “why” and “how” in this use case (of course they are all related in most). As per above, these pricing strategies can be used as an effort to boost sales and / or loyalty, and for a variety of reasons, such as:
etc. Your “when” will have to be determined by the type and the goal of the promotion, plus, you should also take into consideration the budget you can operate with. Some of the pricing strategies will work best in specific times of the year (for e.g.: naturally seasonal promotions and flash sales connected to certain events), while some will need longer time frames to run their course (like loyalty programs). |
Why: |
As mentioned before, your “whys” can be various. One thing however is common among all promotional pricing strategies: when done right, i.e., based on accurate data about product performance and customer buying behaviours, these will let you quickly generate demand for products. Both when they need extra attention as new options arrive and when you need to sell that which remained. Furthermore, the promotion of certain products (or even services) can affect the performance of other products. Customers coming into your store (both online and offline) as a result of a promotion, will often add other items to their baskets, thus analysing the cross-selling effect of your products (detailed in the demand forecasting use case) can further help you in planning your promotions. The “why” in this use case does not only refer to why it makes sense to utilise this tool, but is also a key question for it to work, as you have to decide what the goal of your promotion is before you can specifically choose the time frame, target and other features of your promotion strategy |
How: |
Once you have your “Why” figured out, you can turn your reasons into actionable steps. Like conventional pricing, promotional pricing has its own characteristics, and each business and use case will have specific details that need to be considered when you are implementing promotional pricing. Such as: what has worked for your competition, what results you are expecting from the promotion, what your budget allows you to run, whether your customers are more active online or offline etc. Based on these aspects you can then select a fitting strategy (e.g., loyalty program to grow your customer base and Customer Lifetime Value). The strategy of promotional pricing should always be based on data. A pricing model can be built to help you define and tweak your promo prices. The model should work with purchase data of general sales and previous promo sales, as well as metadata of promotions. Methodologies like price elasticity- and cross elasticity of demand are widely used for promotional pricing. If you’re working in retail, you probably know what these technical terms mean: price elasticity denotes the responsiveness of demand to price changes, while cross elasticity measures how sensitive demand of a product is over a shift of a corresponding product price. These should be your go-to methodologies when trying to define optimal prices – so not only in this one, but most of the pricing use cases. Besides elasticity, other aspects, like:
and other effects (like campaign noise from separate operations) should be taken into account when attempting to define the optimal price. |
Dynamic Pricing
What: |
Capitalise on shifting demand by adjusting your prices in a responsive manner. Utilise dynamic pricing methods for webshops, applications and even offline stores (with preset time frames) to gain both on surge and less demand heavy periods. |
When: |
There are a number of ways you can utilise dynamic pricing, from adjusting your prices automatically with algorithms calculating a shift in demand continuously (surge pricing in high demand periods), through changing prices based on the time of purchase relative to the date of an event (festivals, concerts, sporting events, holidays etc.), to discounted bundles in a set time frame (Buy One Get One Free or BOGOF, happy hour etc.). Keep in mind, that most of these solutions are only applicable to stores that operate an electronic shelf label system. Depending on what your goal is you can use this tool periodically such as connected to seasons as well as on a constant basis such as automatically responding to rises and drops in demand. Based on Hubspot’s great framework, there are 3 boxes to check off when planning to implement dynamic pricing. 1. You have the capacity to gauge how and when demand shifts. In other terms: gather and analyse data about customer behaviour, purchase history, basket composition etc., plus use market best practises etc. to know when demand for certain products / services is likely to shift, and how your customers will likely react to changes in pricing. 2. Your customer base is willing and used to paying fluctuating prices. As mentioned above, changing prices frequently for products / services that customers are not expecting to change will likely result in unhappy customers at best. You definitely don’t want to utilise dynamic pricing for items on a KnownValueItem list (see above in the image driver SKU use case). You can always run experiments with small groups to test customer behaviour. 3. You have sufficient market power or competitors (industry peers) have done it as well / are doing the same. Without being a key choice in your target audience’s mind, your dynamically changing prices can seem unjustifiable and risky. Once you have these boxes checked, how you then implement dynamic pricing strategy will depend on the technological and business context (e.g. an E-commerce store could utilise quickly adjusted prices easier compared to a brick and mortar) . In many cases an advanced Machine Learning service should be developed which can register the ever-changing conversion rates and change the price of previously selected SKUs accordingly |
Why: |
As mentioned above, building a dynamic pricing model on the one hand allows you to respond to changes in demand quickly: rev up when demand skyrockets, then start decreasing the price when it peaks. This helps you to remain competitive and maximise your profits. On the other hand, it can help you generate demand when demand is lower. By creating long term incentives such as set time frames for bundles or payment plans (BOGOF for example), you can generate more traffic in downtimes and likely build customer loyalty |
How: |
Though it has a lot of pros there are some cons you should keep in mind when using the tool of dynamic pricing. Changing your prices too often, based on unclear circumstances (unlike the clarity of surge pricing in say: Uber) can make you seem opportunistic and make your customers feel cheated. To be able to apply dynamic pricing you must be able to accurately assess all factors affecting price elasticity and include these factors into your pricing algorithm. Factors like competitor pricing, product inventory (e.g., purchase history), product trends, seasonality etc. You also have to align your dynamic pricing strategy with your general branding goals. For example, as a luxury brand your dynamic pricing strategy should not focus on offering regular large-scale discounts. On the other hand, as a retailer known for low prices, one should not capitalise on season adjusted, significant price raises as it will likely confuse or even annoy their customers. Most commonly retailers utilise rule-based pricing, which simply put are a set of if [triggering value] - then [previously stored adjustment] rules in a (often in-house developed) software, that relies on the internal knowledge base of pricing managers of the market. Businesses often apply dynamic pricing for reasons that go beyond simply wanting to optimise their prices on a product line for higher profits. The underlying goal might just be that. However, the higher-level goal may be more nuanced. The most common dynamic pricing strategies are:
These then can be further defined with specific rules, created by pricing managers – rules that are most often defined around cost (prices adjusted for a constant margin), competition (adjustments to keep up a certain ratio compared to competition prices), and demand (price increases and reductions adjusted to demand). The issue with internally adjusted, rule-based pricing systems is that they can get overly complex with a wide assortment range (turning almost unmanageable after a point), require frequent manual adjustments (that are often too slow to capitalise on changes) and are based on the know-how relevance of the people monitoring it (hence can’t be expected to react to unpredictable changes). A more feasible approach is the application of Machine Learning based algorithms. The same (and even much more nuanced) strategies may be implemented with significantly less manual adjustment, much larger (and more varied) amount of data processed in a shorter time and more complex “rules” implemented with a significantly lower likelihood of errors. |
Long-term pricing strategy
What: |
Manage customer price-value perception by implementing the most fitting long-term pricing strategy and revisiting your pricing on a systematic basis. Form your customers’ price expectations and value perception in all three stages of the product (service) life cycle: launch, midlife, and late life. As a result, gain higher margins and keep or gain on your market position. |
When: |
A long-term pricing strategy may be beneficial for all businesses, but it is a crucial step for ones with dense competition (and ones with long development phases). Without a properly set up pricing strategy businesses risk losing major investments to R&D and can (and will likely) lower Customer Lifetime Value (i.e.: a customer’s sum value for the business) and lose market share. A broad pricing strategy should be set up at the early stages of the business (better late than never) and specific strategies should be implemented at the introduction of new products (/ services) taking into account your current offerings |
Why: |
Without a pricing strategy that is set up using scenario-based analyses, taking into account internal factors such as new model / version launches and changes in cost position, and external factors such as potential customer and competitors reactions (price changes, product introductions or shifts in customer demand etc.), your prices can be misaligned with your products’ life cycle which can at the bare minimum cause you major revenue loss. In the long run, prices that are not in line with your audience's value perception, and do not take the true costs of serving the market into account, and the right balance of profit and market share, will likely pay with said market share and their market position. Plus, without a long-term pricing strategy, you will not have much luck trying other pricing strategies like dynamic pricing or promo pricing. |
How: |
For one, you should actively manage whether you are prioritising price or volume. Customer value perception and price sensitivity should be of crucial interest throughout the whole life cycle of a product (/service). As the product ages, new products are introduced, or the technology advances the products price has to be aligned. However, it does not mean that the equation is as simple as decreasing product price with time. To be able to set the right price in all three stages (launch, mid-life, late life) of a product, you have to systematically revisit your pricing model, understand your customers’ decision-making processes and the basis of their value perception. Your pricing strategy should always include a broader understanding of your whole assortment range, so that neither do older generations of products anchor the price of your new product lines, nor do new products significantly push down the prices of your older products. In broad terms when setting up a long-term pricing model, it should be based on historical data, machine learning models (which can be built to estimate the demand at any given price point). Plus, optimal prices should always be translated to psychological price barriers in order to maximise margin. |
Logistics And Operations
Operational efficiency with today’s tools can and should go way beyond knowing when certain products are close to a stock-out event. With the right set of data, a value driven business today has to be able to accurately tell:
- What products to sell
- What products will influence the performance of other products
- What features make a product perform better
- How store features will affect assortment performance, and a lot more.
These may seem banal, and to a certain point they are. However, working with data regarding these aspects, and working with the right set of data are two different things. Furthermore, drawing the appropriate conclusions from the data requires expertise, precision, and experience.
Assortment Range Optimization
What: |
Boost your sales, identify crucial product features, understand cross-selling effects and optimise your inventory by analysing customer behaviour and purchase history and specifying an optimal product base for different clusters of your brick-and-mortar stores based on data from your individual locations (plus if you operate online shops in different countries / regions, you should of course optimise your assortment there too). |
When: |
Understanding what makes your customers come into your stores (or onto your online platforms), and what makes them come back, or in other words choose you over your competition should be of crucial interest to you in general. Furthermore, if you operate stores in very different locations (i.e. stores with significantly different demographics / surroundings / environment / size), you should periodically adjust your assortment so it corresponds to local (and seasonal) needs. |
Why: |
As in each use case, your end goal is to optimise your operation by capitalising on the vast amounts of data that is made available by your customers. Whether it’s about creating better online recommendations, sending more appropriate direct mails, tweaking your campaigns, or adjusting your assortment based on customer behaviour/purchase data, the process always involves understanding your customers’ decisions and habits better and tweaking your operation based on the gained insights. For both customer satisfaction and budgetary reasons, stores should adapt to their customer base as both preferences and buying power may significantly vary at different points in time and locations. Finding the optimal assortment range for your stores can boost sales, cut logistics and inventory costs, and reduce production loss and customer churn. |
How: | Usually, this project should start with clustering your stores, as it’s unnecessary and complicated to optimise your stores’ assortment individually (you can read in depth about segmentation in this section). After clustering, you should analyse location specific data to find patterns that reflect similarities between the behavioural patterns and buying decisions of the customer base of different clusters of stores. While in the past, purchasing could simply rely on performance and rotation metrics of the products and the experience of managers in the retail field, with the constantly expanding variety of products and range of competition these simple indicators are no longer sufficient to base well-founded decisions on. The use of advanced analytics (and even AI) is necessary not just for a better understanding of customer behaviour, but to quantify uniqueness and the cross-selling effect of your products, understand the most important factors in increased demand, better forecast demand and more precisely optimise your range. Product range optimization is a continuous effort that should take into account all of the above criteria as new products are introduced and customer choices are changing. Besides cost and margin tweaking it should also align with strategic goals based on your brand’s image (e.g., focus on more fair-trade products). |
Demand Forecasting
What: |
Optimise your inventory, logistics and operations costs, increase customer satisfaction (and through that, loyalty). Build an accurate demand forecasting model so you’ll never have to deal with under- or overstocking again. Demand forecasting is usually a part of assortment optimisation as with a better understanding of your customers’ choices, products’ uniqueness and cross-selling effects, plus with a better understanding of crucial product attributes you’ll have more precise data on product interrelations to base your demand forecasting on. Albeit it is a use case that is viable on its own too, thus why we feel it should have its own detailed “how to” here. |
When: |
As you can see on the graph at the top, this is a high complexity use case. However, it is a tool that every retailer should invest in developing as it has countless benefits, and appropriate stocking is a standard expectation of customers. Whether you own a small business or lead a successful retail chain, a stock-out means a loss of income as well as unhappy customers. As mentioned above in the assortment range optimization use case, knowing simple performance and rotation indicators will not get you far anymore. Without systematically revisiting your assortment’s characteristics and relations, you won’t be able to utilise demand forecasting to its actual potential. |
Why: |
Inventory and logistics optimization drives efficiency and reduces waste. An accurate demand forecasting model is essential to a well-orchestrated inventory and logistics operation plan. Basically, knowing what and when your customers like to buy will help you save costs as you are less likely to overstock or lose business as a result of understocking. An optimised inventory will also help you plan your logistics operations better which also leads to budget optimisation. |
How: |
Start by making sure you have a well maintained, logical inventory. Without accurate data about when and what products your customers are buying your predictions will be useless. It’s even better to have data on when and what you had to restock on, what ended up as excess, what you had to discount etc. If the above is applicable: Use historical data to build predictive models that estimate your customers’ future demand. Keep in mind that although a focus on your best-selling items may seem like a good idea, items that are bought in lower amounts can have major effects on the sellability of your best-selling ones. If a customer is used to buying at yours but regularly faces that a particular item they want to buy is out of stock (or unavailable altogether), they can lower the number of visits or with time switch to a competitor completely. Simply relying on historical purchase data -and nothing else- will lead to understocking as in the case of out-of-stock events (besides perhaps some customer complaints, which might not be recorded) you won’t have data on the actual volume of demand. Consider short term and long-term seasonality of certain products (products connected to holidays, products with a seasonal trend due to recent events – a famous person seen with the product, a paper on the products benefits etc.). |
Store Format Development
What: |
Avoid excess costs and wasted resources by predicting the impact and success of your potential layout remodelling, expansion, modernization, reconstruction and renovation projects. |
When: |
Whenever you are planning on a major property related investment such as opening a new store or considering the layout reorganisation, modernization or reconstruction of existing ones, you should be able to make your decisions based on actual data. This is probably not news to you, but even relatively smaller modifications, such as realigning some of your product displays can massively affect the products’ sales performance. That is why it should not be done without proper analysis on the possible effects. |
Why: |
Because store format development can affect many other parts of your business. Without utilising accurate insight before property development, you can massively decrease sales of certain products. Not surprisingly, predicting the outcome of a business decision can make all the difference: this tool helps you find the right choices in a sea of possible scenarios. |
How: |
For the most accurate forecast, use advanced analytics tools as follows: In case of renovations/reconstruction work, Machine Learning models can be used to predict the effect of the development project. Use before-after sales data of previously renovated/developed stores and locational features to train your model. When you are planning on opening a new location, apply expert models with fine-graded population and traffic data, competitor analysis and purchasing power insights to predict the impact of reconstructions or opening of a new store. |
E-Commerce
When it comes to online shopping, as a retailer you can get accurate data on your customers and audience a lot easier compared to offline tracking. Tracking and targeting your clients can be a piece of cake online.
On the other hand, if you are not doing it right you can just as easily throw your money out of the window both directly and indirectly. If you are not tracking the right metrics, your data will likely be inaccurate, as well as the picture you piece together of your customers.
Even if some of your metrics are on point, you are -hopefully- getting your data from multiple sources (different landing pages, different traffic sources, different platforms etc.), which means that you have to be consistent with the metrics set up. Without that, you can have contradictory data that will not only not help you with your business decisions, but it will also most likely create more uncertainty. Furthermore, without accurate data, your ad campaigns are more than likely wasting resources with little to no effect on your revenue.
At last, it is one thing to have a lot of data funnelled into an analytics tool, and it is an almost entirely other skill to be able to utilise that data.
Online advertisement
What: |
Get better Return On Ad Spend (ROAS): cut ad costs significantly by adjusting your advertisement target group. Segment your customer base by activity, intent, behaviour patterns etc. plus in cases when it makes sense by age, location, etc |
When: |
Can and should be used in a multitude of ways. Among others:
|
Why: |
When used properly, it can be a highly efficient, more accurately trackable way of customer outreach. Thus, testing and tweaking is often easier, with higher efficiency and quicker turnaround. |
How: |
Build. Measure. Learn. This is the cycle to follow with most use cases, but it is perhaps more obviously applicable when it comes to online advertising. There are a large variety of social media sites (and other channels) that a business can use for outreach to their customer base. The key is to first research who your customers exactly are (create an actual persona of your ideal customer – not a vaguely outlined idea, but someone who – based on feedback, current customers & research – resembles an actual customer). Once you have an ideal persona outlined (or a few, such as members of a buying committee in b2b), find out where they spend their time. You can spend all the money in the world on the wrong channel, targeting the wrong people, with a terrible conversion rate, and constantly reiterated messaging. If your ideal customers don’t see it, your testing will have no actionable results. Now that you know who your customers are and know where they hang out: Learn how they talk about your product/service. Or products/services that are direct competitors of yours. Hitting your customers tone of voice and resembling their language in your messaging will work wonders. With all of that, start advertising to them. Don’t try to create the perfect online ad from the get-go. The idea is to try, test, fail, learn, and repeat. |
Product Recommendation
What: | Get better conversions: Recommend products for online shoppers based on product affinity, previous purchases, searches, and similar audiences. Create better user experiences. You can read about a similar use case for your brick-and-mortar stores here. |
When: |
If you’re operating a webshop, or any e-commerce platform (courses, music, movies etc.), this is a must-have. Recommendation engines may seem like a luxury of large players, but with the ever-growing presence of personalization, customers won’t see customised recommendations as an extra, rather as a standard that represents professionalism. Thus, utilising them should not be a question of when as much as a question of how, and what to optimise for? |
Why: |
A recommendation engine allows you to personalise product recommendations, which makes your customers’ experience a hundred times better. As this post by “the good'' collects it: Research has shown that a well-functioning item suggestion system can increase conversion rates by 5.5X, increase purchase likelihood by 75% and by 2018 recommendations accounted for 31% of ecommerce revenues. |
How: |
Get to know your customers’ online behaviour: their browsing history, past activity, previous purchases, abandoned carts, etc. This will help you understand what they are looking for and what they probably like on an online shopping journey, and you will be able to recommend relevant products. You can integrate a recommendation engine to your site and then fine-tune what sort of product recommendation best practises you want to adapt (such as social-proof-based, User Generated Content based, browsing/purchase history based, affinity based etc.). Or you can build your own algorithms with the help of data experts. In any case, personalization (especially a well-built, accurate one) is the sort of “essential” add-on that can still mean a huge advantage at the moment, yet soon will be the standard. It is not only about upselling your customers, but also about creating a better user experience for them while optimising your conversion rates too. |
Offline To Online Conversion
What: |
Improve your sales numbers, lower operational costs, stay in the game by understanding your customer clusters better: learn what types of customers are more likely to shop online and explore the differences between offline and online shopping journeys. Help your offline audience find their preferred products online easier and vice versa help them find the results of their online searches easier offline when they prefer to experience the product in person. |
When: |
Today there are almost no products and services that a customer can’t find an online funnel for. Whether customers find a solution to their problem with an intent-based search, or by chance (and by chance we mean the endless ways people are subjected to product and service promotions and advertisements), more often than not, their customer journey will involve an online element. Even in the case of everyday products, customers are faced with more and more options to handle their shopping online with delivery. In the case of groceries, the aspects (and so the benefits) of offline to online conversion are manifold. For one, it has been observed for a while that most customers prefer supermarkets to hypermarkets with a smaller, more processable assortment range. Online shoppers on the other hand are more likely to have larger baskets with a wider range of products. Turning some of your customers towards your online shops can lead to less store related costs and higher revenue per customer. Thus, the question is: will you capitalise on their online presence, or will you leave your target audience and your hard-earned customers up for grabs for your competition? Not just your direct competition (i.e., other more efficiently optimised brick and mortar stores and new delivery-based retailers), but anything that in the race for audience attention can in one way or another subsidise your products’/services’ place. Offline to online conversion became even more of essence since the global pandemic made customers apparent of the fact: you can, in many cases, leave the in-person experience out of the picture. |
Why: |
If you’re operating a webshop and brick-and-mortar stores as well, it’s essential that you understand what drives some customers to order online and others to stick to offline shopping. Without an accurate picture (i.e., recorded, and processed data) on your customers’ behaviour, and what separates customer segments from one another, you are surely missing out on major opportunities in acquisition. Likely there is a large number of customers who - if informed and approached the right way - are more than happy to buy from your business online, which often will not only mean a change in funnel (i.e., where the customer finishes their buying journey), but an upgrade in their buying behaviour. If the buying process becomes more convenient (the experience is better), plus you can contact customers easier with more personalised offers and reminders, they will be more likely to buy more, more often. |
How: |
First and foremost: get to know your customers better. Explore different behavioural patterns through basket analysis, segmentation, and other methods, and find out why some of your customers are more likely to shop online. Communicate constantly and clearly with your customers about your online platforms. Create a strong online presence that goes beyond simply selling the same products online as offline. Once you have a clear picture on why certain customers are more willing to buy online, or transition to online purchases (as well), you can then leverage that information for other customer segments. Test different approaches in converting your customers to online shopping as well (e.g.: convenience, more personalization, special offers etc.). Measure what worked for certain segments. Learn about different behaviour patterns. Build new customer journeys and restart the loop. |
As a retailer in the 21st century, you cannot (and should not aim to) avoid utilising the never-before-seen amounts of available data at your disposal. As mentioned in the intro of this guide you are likely already on the path of adopting a data driven operation and did not end up on this article by chance.
The aim then is to provide you with a handlebar so you can grip onto something as you are faced with an endless barrage of tips, tricks, “how-to”s and promotions of tools that will solve all of your data analytics needs.
There are many more use cases (some we will update this article within the future), and among all of them are loads that would serve as great pilot projects. Plus, there really are great tools available today that can help you and your BI / analytics teams experiment with retail analytics use cases. However, just as with any industry, there is a reason why there are data analytics consultants (often consisting of experts like data engineers, data scientists, business analysts etc.) who do this professionally.
If your end goal is to be more efficient at capitalising on the data flowing into your business, and sustain growth and competitiveness on the long run, and not just burn through some cash with the type of data analytics that keeps a few busy but has no real effect on your operations, then it makes sense to work on a pilot project with professional retail data consultants.
You don’t have to allocate a huge budget and start with a large-scale use case. You can dip your toes in the shallow waters with something that can run for a few months and will give you quick feedback on why this is crucial if you are aiming on optimising your costs, improving conversion rates, ROAS, CLTV etc., and in general: staying competitive.
One thing is for sure. Market leaders in the retail field are not tackling their analytical needs with a great SaaS tool and a few BI personnel. If they do, they won’t stay leaders for long.
Feel like we are missing something major, or you are looking for more assistance in your data journey? Let us know!
Authors:
Eszter Dudás - Data Scientist
Márton Biró - Data Scientist