Extract URLs From Text

How to extract URLs from text

1. Paste your text into the Your text box, drop a file on it, or use Open file for a .txt, .csv, .html, .xml or .log file. Example loads a short sample.
2. The links appear in the Result box as you type, one per line, with a count such as “3 links” above it.
3. Set the options: Remove duplicates and Add https:// to www. links are on by default; Sort A–Z, Domains only and Only this domain are off.
4. Click Copy, Download .txt, or Use as input to keep working on the list. The links under the result open it in the URL decoder, the duplicate remover or the line sorter.

With the sample text and the default options, the tool returns:

Input:
Read the guide at https://example.com/docs/start?ref=mail (updated), or www.example.org/help.
Mirror: http://files.example.net/v2/setup.zip.

Result:
https://example.com/docs/start?ref=mail
https://www.example.org/help
http://files.example.net/v2/setup.zip

What the tool counts as a URL

A link starts with http://, https://, ftp:// or www. (in any mix of capitals) and runs until a space, a tab, a line break or one of these characters: < > " ' ` { } | \ ^. Those characters are never part of a written address, so links inside HTML attributes, quotation marks and Markdown come out clean. A link also ends at an em dash (—), at an invisible zero-width character (U+200B–U+200D, U+2060, U+FEFF), and at a comma directly followed by another http://, https://, ftp:// or www., so https://example.com/a,https://example.com/b gives two links. The start cannot follow a letter, digit, _, ., @ or -: xhttps://… and abc.www.example.com are not matched.

Then the end of each match is cleaned up:

• Sentence punctuation is removed: a trailing . , ; : ! ? or … and closing curly quotes ” ’ », including several in a row, as in https://example.com/a....
• & becomes &: links copied from HTML source come out usable. Other HTML entities are left as written.
• Unmatched closing brackets are removed: in (see https://example.com/x) the ) is dropped, but in https://en.wikipedia.org/wiki/Mercury_(planet) it is kept because the link opened its own bracket. The same applies to [ and ].

Everything else is kept as written: query strings (?q=1&r=2), fragments (#section), ports (:8080), IP addresses, percent codes (%E2%9C%93) and non-English characters such as https://münchen.de/straße.

Some things are not found: bare domains like example.com, mailto: and tel: links, protocol-relative links (//cdn.example.com/x.js), and relative paths such as /blog/post. For email addresses, use the email extractor.

The options

The steps run in that logical order: links are found and trimmed, https:// is added, links are cut to domains, the domain filter is applied, duplicates are removed, and the list is sorted. Because duplicates are removed after Domains only, ten links to the same site become one line. The count above the result is the number of lines in the final list.

Filtering by domain and listing domains

This sample has five links, one of them repeated:

https://shop.example.com/cart https://www.example.com/blog/post-1
https://example.com/about https://example.org/x https://www.example.com/blog/post-1

A few details of the domain filter:

• Subdomains are included: example.com also keeps www.example.com and shop.example.com.
• Only the host name is checked, as whole labels: example.com drops https://badexample.com/c, and also https://other.com/go/example.com, where the domain appears only in the path.
• Capitals, a leading @, *. or ., a leading https:// and anything after the domain (/path) are ignored, so @EXAMPLE.com, *.example.com and https://example.com/ work like example.com.
• Domains with non-English letters match in either form: münchen.de keeps both https://münchen.de/x and https://xn--mnchen-3ya.de/y. Domains only shows them as written in the text, so https://münchen.de/x gives münchen.de.
• A malformed link the browser cannot read, such as http://[bad/x, is left out when the filter is on.

Links from HTML, CSV files and spreadsheets

HTML source. Paste the page source as it is; you do not need to remove the tags. HTML writes & in attribute values as &, and the extractor reads it back as &, so href="https://example.com/search?q=shoes&page=2" gives https://example.com/search?q=shoes&page=2. Other entities, such as / for /, are copied unchanged; if a link contains them, run the source through the HTML entities decoder first (choose Decode). To read the page text without markup, use Remove HTML tags.

Spreadsheets. Cells copied from Excel or Google Sheets are separated by tabs, which end a link, so a column of addresses pastes cleanly.

CSV files. Commas are allowed inside URLs (map coordinates and some Wikipedia titles use them), so the tool stops at a comma only when another link starts right after it. Two links side by side, https://example.com/a,https://example.com/b, come out as two lines, but in a CSV row such as Name,https://a.com/x,Other the result is https://a.com/x,Other. Open the file in a spreadsheet and copy the column, or replace the commas with spaces using Find and replace before extracting.

Encoded links. Links copied from tracking redirects or search results often contain percent codes such as %2F and %3A. The extractor keeps them as written; paste the result into the URL decoder to read them. The guide to URL encoding explains what each code means.

When people use it

• Checking the links in a draft before publishing an article or newsletter: paste the text and review every address in one list.
• Auditing outbound links on a page: paste the HTML source, tick Domains only to see which sites the page links to.
• Collecting sources from chat logs, meeting notes or exported email threads, with duplicates removed.
• Cleaning server or app logs: open the .log file and keep only the links to one domain with Only this domain.
• Building a list for another tool, such as a crawler or a redirect sheet, by downloading the result as a .txt file with one URL per line.

For large files and for extracting emails and links together, see how to extract emails and URLs from large text files.

Questions