Customer Segmentation with RFM Analysis and K-Means: A Retail Marketing Pipeline

TL;DR: An open-source retail marketing analytics pipeline that turns raw transaction logs into actionable customer segments: synthetic transaction data generation (20,000+ transactions, 1,000 customers), RFM (Recency, Frequency, Monetary) feature engineering, log-transform + scaling + K-Means clustering (k=4), 3D and pairplot visualizations, and an auto-generated markdown segment report. One command, Docker included.

1. The problem: not all customers are the same

Every retailer knows this intuitively, but most marketing still treats the customer base as one blob: same email blast, same discount, same timing. The cost is real — high-value loyal customers get the same generic 10%-off as one-time bargain hunters, while at-risk customers quietly stop buying and nobody notices until they're gone. Segmentation is the fix: group customers by behavior, then market to each group differently — win-back campaigns for the at-risk, VIP treatment for the loyal, onboarding for the new.

The classic framework for this is RFM analysis: Recency (how recently did they buy?), Frequency (how often do they buy?), Monetary (how much do they spend?). Three numbers that capture most of what a marketer needs to know about a customer. This project computes RFM from transaction data and lets K-Means discover the segments automatically.

2. What the project is

retail-marketing-analytics is a Python pipeline: generate synthetic retail transactions, compute RFM per customer, cluster with K-Means, visualize the clusters, and write an interpretable segment report. Everything from raw (simulated) invoices to a marketing-ready segment description in one run.

Project structure (verified from the repo tree):

3. The data: 20,000 realistic transactions

data_generator.py simulates an online-retail transaction log in the classic format — InvoiceNo, StockCode, CustomerID, Quantity, InvoiceDate, UnitPrice — with seasonality and varying purchasing power baked in, so the data has the lumpiness real retail data has: holiday spikes, whales and minnows, customers who buy once and vanish. Twenty thousand-plus transactions across 1,000 customers is enough for the clusters to be stable rather than noise.

4. How it works: RFM, then K-Means, then a report a marketer can read

segmentation.py runs four stages:

1. Cleaning. Cancellations (negative quantities) and rows with missing CustomerID are filtered out — the two most common garbage sources in retail transaction exports. TotalPrice is computed as Quantity × UnitPrice.
2. RFM computation. Per customer: Recency = days since last purchase (against a snapshot date one day after the final transaction), Frequency = number of unique invoices, Monetary = total spend. One table, 1,000 rows, three columns.
3. Clustering. RFM distributions are heavily right-skewed (a few customers spend enormously), so the pipeline applies a log1p transform before StandardScaler — an important, correct choice: K-Means is distance-based, and without de-skewing, the whales would dominate the geometry. Then KMeans(n_clusters=4, random_state=42, n_init=10) assigns each customer to one of four segments.
4. Visualization and reporting. A seaborn pairplot and a 3D scatter of the scaled RFM space show segment separation, and the script writes segmentation_report.md with per-cluster means and sizes.

Here is what the generated report actually says:

The interpretation writes itself: Cluster 3 is the gold tier (bought recently, buys constantly, spends 2.4× the next group); Cluster 2 is the at-risk/lost group (hasn't bought in ~100 days, low frequency, low spend); Cluster 1 is the active core; Cluster 0 is the broad middle. Each maps to a different marketing play: VIP retention, win-back, engagement, and nurture, respectively.

The log-transform before scaling is the detail that separates a working segmentation from a broken one. Retail monetary data is so skewed that raw K-Means mostly discovers "the three whales and everyone else." Log-space discovers actual customer archetypes.

6. From segments to campaigns: what each cluster is for

Segments are only worth what the marketing team does with them. The four clusters map directly to different plays:

The visualizations support the same story. The 3D scatter of the scaled RFM space (rfm_3d_plot.png) shows the clusters pulling apart cleanly — the gold cluster sitting high on all three axes, the at-risk cluster sunk on recency — while the pairplot (rfm_clusters_pairplot.png) confirms the separation holds in every two-dimensional projection, not just in 3D. That matters: clusters that separate in 3D but blur in pairwise plots are often an artifact of the projection, not real customer archetypes.

The strategic insight hiding in the numbers: Cluster 0 is the largest group (518 customers) spending $30K on average with 12 orders each. Moving even a fraction of them toward Cluster 1 behavior — one more purchase per quarter — is likely worth more than any win-back campaign, because the middle is where the volume lives.

7. How to run it

git clone https://github.com/Shivay00001/retail-marketing-analytics.git
cd retail-marketing-analytics
pip install -r requirements.txt   # pandas, numpy, scikit-learn, matplotlib, seaborn
python data_generator.py          # builds online_retail_simulated.csv
python segmentation.py            # RFM + KMeans + plots + segmentation_report.md

Docker support is included. The output artifacts — the two PNGs and the markdown report — land in the working directory, ready to drop into a slide deck.

8. Honest status and roadmap

What this is: a complete, working segmentation pipeline with correct methodology (cancellation filtering, log de-skewing, seeded reproducible clustering, interpretable auto-reporting). The four segments are stable, the report is marketer-readable, and the code runs end to end.

What it isn't: it trains on synthetic data, and k=4 is assumed rather than selected — the code comment says so ("Assuming K=4 for simplicity"). That's an honest simplification, but it's also the first thing to fix. Same README caveat as the sibling repos: the listed BI stack (R, PowerBI, Tableau, Qlik Sense, BigQuery, SAS) isn't exercised by the Python code; the real stack is scikit-learn, pandas, and matplotlib/seaborn.

The roadmap: select k empirically with the elbow method and silhouette scores (or let DBSCAN find the natural density structure); add customer lifetime value modeling as a second axis beyond raw monetary; track segment migration over time (customers moving from gold to at-risk is the early-warning signal that matters); and connect the segments to actual campaign tooling — a segment is only worth what the email platform does with it.

One more discipline worth adding: re-run the segmentation on a rolling window rather than once. Customer behavior drifts with seasons, promotions, and life events, and a segment map from six months ago will misclassify exactly the customers who changed most. A monthly refresh — same code, fresh transactions — keeps the archetypes honest and catches migration while there's still time to intervene.

Frequently asked questions

What does the retail-marketing-analytics repo do?

It segments retail customers using RFM analysis (Recency, Frequency, Monetary) plus K-Means clustering, generating visualizations and a marketer-readable report with four interpretable segments — from gold-tier loyalists to at-risk customers.

Why log-transform before clustering?

Retail spend data is heavily right-skewed. K-Means is distance-based, so raw values let a few big spenders dominate the geometry; log1p compression plus scaling lets the algorithm find real customer archetypes.

Is k=4 the optimal number of segments?

It's assumed for simplicity, per the code comments — not selected empirically. Elbow/silhouette analysis would be the next step for a production deployment.

Is the data real?

No — 20,000+ synthetic transactions with simulated seasonality. The methodology (cleaning, RFM, clustering, reporting) transfers directly to real transaction exports.

The bottom line

Segmentation only matters if a marketer can read the output and act on it. This pipeline goes from invoices to a four-segment report with names, numbers, and next actions — with the statistical hygiene (de-skewing, seeded clustering, reproducible runs) that keeps the segments honest.

Repo: github.com/Shivay00001/retail-marketing-analytics — MIT licensed. Want customer segmentation on your real sales data? VisionQuantech builds marketing analytics for retail and e-commerce. Contact: [email protected]