Term Frequency — Inverse Document Frequency

$$TF_{ij} = \frac{f_{ij}}{n_j} \\ IDF_{i} = 1 + \log(\frac{N}{c_i}) \\ TFIDF_{ij} = TF_{ij} \times IDF_{i}$$

Where:

• $f_{ij}$ is the frequency of term $i$ in document $j$
• $n_j$ is the total number of terms in the document $j$
• $N$ total number of documents
• $c_i$ is the number of documents containing word $i$

TF-IDF is improvement over “bag of words” - frequently repeated words (like “stop wrods”) have less weight.

Example

Original documents:

1. abc abc def ghi
2. abc def ghi jkl
3. ghi jkl mno pqr

Converted to term frequency vectors (absolute):

	abc	def	ghi	jkl	mno	pqr
1	2	1	1	0	0	0
2	1	1	1	1	0	0
3	0	0	1	1	1	1

Converted to term frequency vectors (relative):

	abc	def	ghi	jkl	mno	pqr
1	0.5	0.25	0.25	0	0	0
2	0.25	0.25	0.25	0.25	0	0
3	0	0	0.25	0.25	0.25	0.25

Document frequencies:

	abc	def	ghi	jkl	mno	pqr
DF	2	2	3	2	1	1
IDF	1.17..	1.17..	1.0	1.17..	1.47..	1.47..

TF-IDF:

	abc	def	ghi	jkl	mno	pqr
1	0.585	0.2925	0.25	0	0	0
2	0.2925	0.2925	0.25	0.2925	0	0
3	0	0	0.25	0.2925	0.3675	0.3675

Finally TF-IDF can be used together with Cosine similarity.