Programming for Applications

Chapter 22: Machine Learning

Yu-You Liou (NTU)

Shih Chien University

2026-07-20

Finding Structure

Beyond Supervised Learning

Chapters 20–21 were supervised — predict a known response. This chapter covers two unsupervised tasks, where there is no response to predict, only structure to discover:

  • market basket analysis — which items occur together (association rules);
  • clustering — which observations are similar enough to group.

Market Basket Analysis

Association Rules and apriori

Given a set of transactions (a shopping basket, a listening history), association-rule mining finds items frequently associated with each other — “customers who bought X also bought Y.” The classic algorithm is a priori, in package arules:

library(arules)
apriori(data, parameter = NULL, appearance = NULL, control = NULL)
  • data — a transactions object (or a matrix/data frame coercible to one);
  • parameter — thresholds, chiefly support (how often items appear together) and confidence (how reliable the rule is);
  • appearance constrains which items may appear on each side; control tunes the algorithm.

arules is built on S4 classes (a transactions class for the data) and the well-engineered apriori C implementation by Christian Borgelt.

Example: Audioscrobbler

The book mines Audioscrobbler listening data (now part of Last.fm). Load with read.transactions, mine, inspect:

library(arules)
playlists <- read.transactions("audioscrobbler.csv", format="basket", sep=",")
rules <- apriori(playlists,
                 parameter=list(support=0.01, confidence=0.5))
inspect(head(sort(rules, by="lift"), 10))   # strongest rules by lift

Resulting rules read like “listeners of Green Day and Nirvana also listen to Red Hot Chili Peppers,” each with support, confidence, and lift (how much more often than chance). A related, faster algorithm for frequent itemsets is eclat.

Clustering

Distance Measures

Clustering groups observations so that members of a cluster are more similar to each other than to outsiders — and similarity starts with distance. stats::dist builds a distance matrix:

dist(x, method = "euclidean", diag = FALSE, upper = FALSE, p = 2)

method may be "euclidean", "manhattan", "maximum", "canberra", "binary", or "minkowski" (with parameter p). The right metric matters: scale and units shape every cluster that follows, so standardize variables first when they differ in range.

d <- dist(scale(iris[, 1:4]))     # standardize, then Euclidean distance
as.matrix(d)[1:4, 1:4]
         1         2         3         4
1 0.000000 1.1722914 0.8427840 1.0999999
2 1.172291 0.0000000 0.5216255 0.4325508
3 0.842784 0.5216255 0.0000000 0.2829432
4 1.100000 0.4325508 0.2829432 0.0000000

Partitioning: kmeans, pam, clara

Partitioning algorithms split data into a chosen number k of clusters:

  • kmeans(x, centers, ...) — fast, minimizes within-cluster sum of squares; sensitive to outliers and starting centers.
  • cluster::pam(x, k) — partitioning around medoids; uses actual data points as centers, more robust than k-means.
  • cluster::clara(x, k) — pam scaled to large data via sampling.
set.seed(1)
km <- kmeans(scale(iris[, 1:4]), centers=3)
table(km$cluster, iris$Species)   # clusters vs. true species
   
    setosa versicolor virginica
  1      0         39        14
  2      0         11        36
  3     50          0         0
km$centers[, 1:2]
  Sepal.Length Sepal.Width
1  -0.05005221 -0.88042696
2   1.13217737  0.08812645
3  -1.01119138  0.85041372

Cluster Quality and Hierarchical Clustering

  • cluster::silhouette measures how well each point fits its cluster (−1 to 1); the average silhouette width helps choose k.
  • Hierarchical clustering needs no k up front — hclust(dist_object, method=) builds a tree (dendrogram), merging closest clusters step by step; cluster::agnes (agglomerative) and diana (divisive) are richer alternatives.
hc <- hclust(dist(scale(iris[, 1:4])), method="ward.D2")
plot(hc, labels=FALSE, main="Hierarchical clustering of iris")
rect.hclust(hc, k=3)              # cut into 3 clusters

method ranges over "complete", "average", "single", "ward.D2", … — each defining “distance between clusters” differently, and each producing differently shaped clusters.

Tip

Unsupervised work has no answer key. There is no accuracy to optimize — only structure to judge. Vary the distance metric, the algorithm, and k; validate with silhouette widths, domain knowledge, and whether the clusters are useful, not merely whether they exist.