Text Cleaner, Sanitizer & Formatter
Strip HTML tags, normalize multiple spaces, eliminate blank lines, remove emojis, and format text case seamlessly.
Unlike conventional calculator sites that send your financial numbers, payload data, or code snippets across remote web servers, Affi-Chain processes all mathematical derivations, algorithms, and data parsing 100% locally within your browser sandbox. No tracking cookies, no server-side retention, and zero transmission lag.
What is Text Cleaning & Formatting?
A Text Cleaner & Formatter is an automated text-processing utility designed to scrub, normalize, deduplicate, and sanitize messy, unstructured text corpora. In modern data science, natural language processing (NLP), web scraping, digital marketing, and content operations, raw text ingested from OCR document scans, PDF extracts, database exports, and scraped HTML is fraught with formatting anomalies:
- Erratic whitespace, mixed tabs, and dangling carriage returns.
- Non-standard Unicode characters, zero-width spaces, and broken diacritical marks.
- Lingering HTML/XML markup tags, script blocks, and encoded entities.
- Duplicate lines, empty rows, and inconsistent letter casing.
Before raw text can be passed to Large Language Models (LLMs), indexed inside Elasticsearch, analyzed in Python data pipelines, or typeset in publishing workflows, it must undergo systematic normalization.
The Affi-Chain Text Cleaner & Formatter provides high-speed, 100% in-browser text sanitization, executing regex transforms, whitespace collapsing, line deduplication, and case normalization locally with zero latency and complete privacy.
Core Text Transformation Operations & Transformation Matrix
The cleaning engine provides granular controls over common string manipulation tasks:
| Operation | Input Pattern | Sanitized Output | Underlying Regex / Method | Primary Use Case |
|---|---|---|---|---|
| Strip HTML Tags | <p>Hello <b>World</b></p> | Hello World | /<[^>]*>/g | Preprocessing scraped web text for LLM ingestion |
| Collapse Whitespace | Hello World | Hello World | /[\s\u00A0]+/g → ' ' | Normalizing irregular spacing from PDF/OCR copies |
| Remove Empty Lines | Line 1\n\n\nLine 2 | Line 1\nLine 2 | /^\s*[\r\n]/gm → '' | Eliminating vertical bloat in logs and datasets |
| Deduplicate Lines | Apple\nBanana\nApple | Apple\nBanana | Set lookup (preserves order) | Cleaning email lists, keyword sets, and CSVs |
| Strip Non-ASCII | Café Résumé 🚀 | Cafe Resume | Unicode NFKD decomposition | Legacy ASCII systems, POSIX terminals, SMS gateways |
| Sentence Capitalization | hello world. it works. | Hello world. It works. | Period boundary scanner | Standardizing unformatted customer submissions |
Step-by-Step Practical Transformation Example
Let us examine how a corrupted text excerpt scraped from an online catalog with mixed HTML, duplicate rows, and excessive whitespace is cleansed:
Raw Input Corrupted Text:
<div class="product-card">
<h2> SKU-8820: Organic Ethiopian Coffee </h2>
<p>Notes of dark chocolate, berry, and jasmine.</p>
</div>
SKU-8820: Organic Ethiopian Coffee
Notes of dark chocolate, berry, and jasmine.
Price: $18.50 per 12oz bag.
Price: $18.50 per 12oz bag.
Cleaned & Deduplicated Output:
SKU-8820: Organic Ethiopian Coffee
Notes of dark chocolate, berry, and jasmine.
Price: $18.50 per 12oz bag.
Practical Code Implementation: High-Speed Client-Side Text Engine
Below is a modular JavaScript utility that applies configurable sanitization passes to any text payload:
/**
* Enterprise In-Browser Text Sanitization Engine.
*/
class TextCleaner {
/**
* Cleans text using configurable boolean flags.
*/
static clean(text, options = {}) {
if (!text || typeof text !== 'string') return '';
let result = text;
// 1. Strip HTML/XML tags
if (options.stripHtml) {
result = result.replace(/<[^>]*>/g, '');
}
// 2. Normalize Unicode (NFKC handles ligatures and compatibility characters)
result = result.normalize('NFKC');
// 3. Remove zero-width spaces and soft hyphens
result = result.replace(/[\u200B-\u200D\uFEFF\u00AD]/g, '');
// 4. Normalize non-breaking spaces to standard spaces
result = result.replace(/\u00A0/g, ' ');
// 5. Line by line processing
let lines = result.split(/\r\n|\r|\n/);
// Trim line edges
if (options.trimLines !== false) {
lines = lines.map(line => line.trim());
}
// Collapse multiple internal spaces per line
if (options.collapseSpaces) {
lines = lines.map(line => line.replace(/[ \t]{2,}/g, ' '));
}
// Remove empty blank lines
if (options.removeEmptyLines) {
lines = lines.filter(line => line.length > 0);
}
// Deduplicate identical lines (preserving first occurrence order)
if (options.deduplicateLines) {
const seen = new Set();
lines = lines.filter(line => {
if (seen.has(line)) return false;
seen.add(line);
return true;
});
}
return lines.join('\n');
}
}
// Verification Example:
const messyText = " <div>First Item</div> \n\n <div>First Item</div> \n Second Item ";
const cleaned = TextCleaner.clean(messyText, {
stripHtml: true,
collapseSpaces: true,
removeEmptyLines: true,
deduplicateLines: true
});
console.log("Sanitized Output:\n" + cleaned);
// Output:
// First Item
// Second Item
Comparison Matrix: Text Cleaning Paradigms
| Tool / Paradigm | Execution Speed | Memory Overhead | Browser Availability | Regex Support | Best Use Case |
|---|---|---|---|---|---|
| Affi-Chain Client Cleaner | Instant (Zero latency) | Negligible | 100% Native Browser | Full ES2024 Regex | Ad-hoc data cleaning, email lists, content staging |
Unix Shell (sed, awk, tr) | Ultra-Fast | Low | Terminal / CLI only | POSIX ERE | Batch processing multi-gigabyte server log files |
Python (re, unicodedata) | Very Fast | Moderate | Python Runtime required | Advanced Python regex | Automated data engineering & ETL pipelines |
| Spreadsheet Formulas (Excel) | Slow | High | Office Suite | Limited formula functions | Casual business analysts cleaning tabular rows |
Critical Edge Cases & Text Processing Pitfalls
1. Naive HTML Stripping Breaking Mathematical Expressions
A common disaster occurs when naive regex tag-strippers (/<.*?>/g) process text containing comparison operators:
- Input:
Condition: if (x < 5 && y > 10) return true; - Broken Output:
Condition: if (x return true;Why this happens: The greedy regex assumes< 5 && y >is a single HTML element and erases all characters between<and>. Solution: Use an explicit HTML parser or a regex requiring an immediate alphabetical tag name:/<(?:\/)?[a-zA-Z][a-zA-Z0-9]*(?:\s+[^>]*)?>/g.
2. Hidden Unicode Zero-Width Spaces
Characters like \u200B (zero-width space), \uFEFF (zero-width no-break space), and \u00A0 (non-breaking space) are visually identical to ordinary spaces or invisible altogether. Standard regex \s in some older environments does not match all Unicode space separators, allowing invisible characters to disrupt database unique index constraints. Always execute Unicode NFKC normalization and explicit code point removal.
3. Preserving Desired Line Breaks in Markdown / Code
Aggressive whitespace collapsing often destroys intended paragraph structures and source code indentation. When cleaning programming code or Markdown documents, disable collapseSpaces to avoid flattening functional indentation.