Customer Relationship and Sales Management using Data Mining Techniques

A general store is a destination for customers to buy everything they require in a single spot. The Manager is responsible for administering and making decisions for developing and running the store. But it is a heavy task for a single person to make valuable decisions that do not affect the business. So, in this project, data mining techniques are applied to the store’s transactional data to help the manager make decisions. We use RFM, Recency Frequency and Monetary segmentation method and classification algorithms to classify the customers into loyal and not-so-loyal customers to make predictions based on the data of the valuable customers. The customers who are loyal to the store are selected and their data is used for further analytics. Association mining technique to obtain the products which are most likely to be bought together and other data mining and visualization techniques to display the valuable knowledge mined from the dataset. Then we cluster the products based on the product’s important keywords and then the customers are further classified by finding the number of products bought from the previously found clusters.


Introduction
Stores that would sell various kinds of products that the customer may require are called General Stores or Supermarkets. Due to the growing population and the increasing requirements of customers from General Stores, it has become tough for the general stores to attain the credibility of their customers. As the reliance on General Stores has become high for the people, it has become the general store's responsibility to know their customer's requirements and to attain them. For the successful management of the General Stores, it is needed to make the right decisions at the right times, which is invariably being done by the store manager, the accuracy of the decisions made is limited to the individual's ability and there is a high possibility of subjective errors. To reduce the risks due to the above-mentioned factors, it is required to automate the decision-making processes of General Stores using data mining algorithms.

Goal
Understanding our customers and making decisions based on their requirements will be enabling the manager to make appropriate decisions for the General Stores processing. This model intends to find out the customers who are more impactful in the sales of the stores and find out the insights about their purchase pattern. Making such refinement of the customer will reduce the gap between the actual occurrence and the perception of the sales in-store. Making the appropriate perception will lay a path for deciding on things that would be having a greater impact on the store's progression. Dataset consists of attributes of every purchase that are made in the store, which includes detail of the customer who made the purchase, the product details. Upon the dataset, the customers with high relevance called as "loyal" customers are found out with their purchase details mapped along. With the refined details, further segmentations are made to find out the insights.

Literature Survey
With the increasing number of competitions for a general store, the store manager must make valuable decisions for the store to attract new customers and to retain existing customers. But with a large number of customers, the task of making business decisions that is suitable for the customers as well as for the development of the business is almost impossible for a single manager. This paper tells the methods and the techniques which can be employed for easing the manager's workload.
[1] [2] The best way of doing this is to segment the customers based on their loyalty towards the store. [3] A business principle -'80/20 rule' states that 80% of a business's profit is from 20% of its customers [3]. So, it is enough to target the loyal customers for the developing prediction models for the development of the business instead of targeting every customer. [1] Also we have to understand that not all customers who spend much in a store are loyal customers and not all customers who spend less in a store are disloyal customers [1]. [4][5][6][7] We can segment the customers based on their loyalty into two segments -The Loyal Customer and The Not-So-Loyal Customer using the RFM Segmentation technique. RFM segmentation is a method where a customer is given a score based on how recently they visited the store -Recency, how frequently they visit the store -Frequency, and how they spend in the store -Monetary [4] [7]. Based on their RFM score the loyalty is decided by keeping a threshold loyalty score. Customers above the threshold are considered to be Loyal and the rest as Not-Loyal. Now the data of the Loyal customers are considered for the rest of the data mining process [3].
[8] Products sold in the store are initially clustered based on their properties. The Loyal customers are further classified based on the cluster from which they have bought the most number of products in that store [9]. So, the number of classifier labels depends on the number of product clusters formed previously.
[10] The loyal customers' data is again used to find the association or relationship between the products. [11] We can get information about the products which are often bought together by the customers. Using this data, the store manager can arrange these products on the shelves nearby which increases the probability of selling these products to a single customer.

RFM Classification
Customers of a General Store may differ drastically from one another, some may have a greater contribution to the store's sale and others may have a much lesser contribution. To make decisions based on customer buying patterns at first it is required to find whom to be considered and whom to be left behind, to do these we use a model called as R-F-M (Recency -Frequency -Monetary) model. Which would consider recency, frequency and monetary contribution of each customer to derive an RFM score for every customer, based on the RFM score the customers are classified into loyal (who are considered for decision making) and not-so-loyal (who are not considered for decision making). We generate a classification model to predict the loyalty of future customers.

Product Segmentation
For making decisions in General Stores, it is required to consider the product which is having a pattern in the customer's purchase history, those products can be considered to complement each other or belonging to a particular set of purchases. This process of clubbing products that are relevant to each other is called segmentation. The products are segmented so that decision making for a single product in the segment will work out in a similar way collectively for other products that are present in the same segment as the former.

Customer Classification
Now the loyal customers are once again segmented based on the products they buy. A customer might have bought many products. Every individual product that a customer bought will fall under any one of the product segments. The products under which segment the customer mostly bought are founded out and it is used to segment the customers. A classification model will be developed to segment the future customers.

Product Association
The loyal customers' purchase data is used to find the association or relationship between the products in the store. By doing so we can find the products which are often bought together. Using this data, the store manager can keep the products with high relationships together for display in the store so that the customers might get what they wanted in a single location without searching for them in different locations.

2.Case Study
A case study is done using a general stores dataset.

Data Set
This is a transactional data set which contains all the transactions that occurred in an wholesale all-occasion gift selling store in the UK. This is a multivariate, Sequential, Time-Series data set with 541909 records and 9 attributes respectively. Data set consists of the following attributes

Pre-Processing
The dataset needs to be cleaned otherwise it may lead to inconsistent outcome. The dataset which we use consists of around 0.2% of null values in the description column and 24.9% of null values in the Customer ID column. The record with these null values is removed. Fig. 1 and Fig. 2 represent the before and after of removing the records with null values.
Certain records in the dataset are duplicates. These records are removed from the dataset. The dataset consists of certain records which represent the orders which are cancelled. These records are identified by their invoice number which have the letter 'C' followed by the actual order initially placed by the customer. These records are removed along with their counterparts. In certain cases, the records for cancelled orders might not have the counterpart, they are also identified and removed.

RFM Segmentation
The recency, frequency and the monetary values are extracted from the dataset and stored in a separate data frame (Fig. 3) . The quartile values of the above attributes are also found with respect to the values 0.25 , 0.50, 0.75 (Fig. 4) .   The loyalty of the customer is decided based on the RFM score. For this dataset, the threshold for the loyal customers is set to 222. Customers with the RFM score above 222 are said to be the loyal customers and the rest of the customers are not so loyal to the store (Fig. 6). Now the distribution of the loyal and the non-loyal customers is displayed in scatter plot representation (Fig.7).

Fig.6 Customer Loyalty data
Now to find the loyalty for future customers, we use a classification model. The Recency, Frequency and Monetary values of the customers are used as feature and the Loyalty as the label for the classification model (Fig. 8). The Decision Tree classification algorithm is used to train the model and achieve 99 % accuracy.
Concise the loyal customer into a separate dataframe, which would be used for further processing.

Product Segmentation
The product description is now used for grouping the products (Fig. 9) . The important keywords are used for the clustering process. K-Means clustering algorithm is used to group the products. Five clusters of products are formed from the dataset.

Customer Classification
The customers are classified based on the products they purchase. To do this, we need to find the respective cluster the customer prefers the most. This data is now used to develop a prediction model to predict the product a customer might buy in the future. Random forest classification algorithm is used to train the prediction model.

Product Association
The products which are bought together in the most transactional cases are found using apriori association algorithm. The support, confidence and lift values are used to identify the rules which are more likely to occur in real life ( Fig. 10).