Skip to main content

Build a Text Pipeline

A Text Pipeline transforms text locally in your browser. Each layer receives the previous layer's output. The last layer produces the final output.

Build a pipeline from raw logs: keep errors, redact addresses, remove duplicates, and sort. Compare the changes and inspect the final result.
Free
Included: two transformation layers
Pro
Included
Self-hosted
Optional add-on
Compare plans

The pinned Input tab does not count toward the layer limit.

Create a pipeline​

  1. Select Text Pipeline in the left navigation rail.
  2. Paste text into Input, or select browse files to load a file.
  3. Select Add layer (+) beside the tabs.
  4. Choose an operation or template from Text, Regex, Redact, Extract, or Filter.
  5. Configure the layer and inspect its output.
  6. Add another layer to transform that output.

The Input tab holds your editable source text. Transformation tabs show read-only input and output. To load files later, open Input > Tab actions > Load files…. See file loading for formats and access.

Choose operations and templates​

Text includes Trim lines, Unique lines, Remove empty lines, Sort lines, Case transform, Limit lines, and Normalize Unicode. It also includes templates to remove ANSI colour codes, collapse spaces and tabs, normalize line endings, or split simple lists.

Regex provides three operations:

  • Regex substitution replaces matches and keeps unmatched text.
  • Regex extraction produces output from matches, using your extraction template. No matches produce an empty string.
  • Filter lines keeps or removes whole lines that match a regex.

Redact and Extract provide templates for IP addresses, email addresses, MAC addresses, and credit-card candidates. Extract also includes UUIDs, ISO dates, HTTP(S) URLs, and hash candidates.

Filter contains NGINX templates. Error, warning, and info templates use the standard text error log. Keep NGINX errors includes error, crit, alert, and emerg entries. The 4xx and 5xx templates use the status field in common or combined access logs.

A template creates an editable regex layer with a predefined flavor. Select Add layer > Redact > Redact IPv4 to replace addresses with [IP].

Add Redact IPv4 after the filter. Four addresses become redaction markers.

Redaction templates are starting points: inspect the output before sharing it. Credit-card and hash matches are candidates, not verified values. ISO date extraction checks numeric ranges, not calendar validity.

Use Trim lines before Remove empty lines to remove whitespace-only lines too. Line operations recognize LF, CRLF, and CR. Use Normalize line endings to LF when you need one line-ending format. Split on commas and semicolons handles simple lists, not quoted CSV fields.

Change layer settings​

The controls below the tabs show settings for the selected operation:

  • Sort direction selects Ascending or Descending.
  • Case selects Uppercase or Lowercase.
  • Filter mode selects Keep matches or Remove matches.
  • Keep selects the first or last number of lines. The default is the first 10 lines.
  • Normalization form selects NFC, NFD, NFKC, or NFKD. The default is NFC.

Tab actions also contains operation choices. Set the line count in the layer controls. Regex layers also have their own flavor, pattern, flags, and replacement or extraction template.

To switch operations, open Tab actions > Operation. Regex layers can switch between Substitution, Extract, and Filter lines. Text layers can switch between text operations. Shared fields stay unchanged. Operation-specific settings reset to their defaults. Switching to Filter lines discards the replacement or extraction template.

Switch an IPv4 regex from substitution to extraction. Use $&\n to return each address on a separate line.

Organize layers​

Select a tab to inspect its layer. Double-click its name to rename it. Drag transformation tabs to reorder them. Right-click a tab, or open Tab actions (⋮), to disable or delete it. Add layer after inserts a layer directly after that tab.

Insert Unique lines between redaction and sorting. The following layer receives the updated output.

The Input tab stays first. A disabled transformation layer passes its input through unchanged. Drag the dividers between editor areas to adjust their heights.

Disable Unique lines to pass duplicates through. Enable it again to restore the three-line result.

Pipeline actions > Clear pipeline removes the input and all transformation layers after confirmation. Save a version first if you need to keep them.

Compare a layer's changes​

In the output heading, open View: Output and select Changes. The comparison shows input on the left and output on the right. Red marks removed text. Green marks added text. Unchanged sections collapse to show context. An unchanged result shows No changes.

Switch from Output to Changes to compare this layer's input and output side by side.

Large or complex comparisons can show changed blocks instead of individual character changes. Select Output to return to the result.

Run and inspect the pipeline​

Pipeline actions > Run pipeline starts a run. You can also press Cmd + Enter on macOS or Ctrl + Enter elsewhere. During a run, select Cancel pipeline or press Escape to stop it.

To run after edits automatically, enable Run pipeline automatically under Settings > Matching > Automation & benchmarks. The manual run action remains available when automatic execution is on.

Previous output can remain visible while a new result is pending. Copy actions are disabled for outdated results.

The right details panel shows the Execution trace. Select a row to open its layer. Each row reports input and output line counts, character counts, time, and an operation-specific effect. For example, Filter lines reports kept and removed lines. Unique lines reports duplicates removed. Substitution reports replacements. Hover over a layer's status icon to read its status or error.

If a layer fails, later layers cannot complete. Select the affected row, correct the error, and run again. If layer data cannot load, use Run pipeline to regenerate it. Verify Pipeline completed before copying the final result.

Copy, save, and share​

Select Pipeline actions > Copy final output to copy the completed result from any tab. Use a layer's Copy… > Input or Copy… > Output menu to copy its text. The read-only editors also have copy buttons at the top right.

Save the workspace to keep its input and layer configuration. Save before opening another workspace or reloading the page. If a pipeline exceeds your plan's layer limit, delete extra layers to make it editable.

Publish a pipeline or find one in the Community Library.