urllib.parse should sanitize urls containing ASCII newline and tabs.¶
A security issue was reported by Mike Lissner wherein an attacker was able
to use \r\n
in the url path, the urlparse method didn’t sanitize and
allowed those characters be present in the request:
>>> from urllib.parse import urlsplit
>>> urlsplit("java\nscript:alert('bad')")
SplitResult(scheme='', netloc='', path="java\nscript:alert('bad')", query='', fragment='')
Firefox and other browsers ignore newlines in the scheme. From the browser console:
>> new URL("java\nscript:alert(bad)")
<< URL { href: "javascript:alert(bad)", origin: "null", protocol:
"javascript:", username: "", password: "", host: "", hostname: "", port: "", pathname: "alert(bad)", search: ""
Mozilla Developers informed about the controlling specification for URLs is in fact defined by the “URL Spec” from WHATWG which updates RFC 3986 and specifies that tabs and newlines should be stripped from the scheme.
See: https://url.spec.whatwg.org/#concept-basic-url-parser
That link defines an automaton for URL parsing. From that link, steps 2 and 3 of scheme parsing read:
If input contains any ASCII tab or newline, validation error. 3. Remove all ASCII tab or newline from input.
urlparse module behavior should be updated, and an ASCII tab or newline should be removed from the url (sanitized) before it is sent to the request, as WHATWG spec.
More commits:
Doc changes:
Dates:
- Disclosure date: 2021-04-18 (Python issue bpo-43882 reported)
- Reported at: 2021-03-16 (email sent to the PSRT list)
- Reported by: Mike Lissner
Fixed In¶
- Python 3.6.14 (2021-06-28) fixed by commit 6c472d3 (branch 3.6) (2021-05-06)
- Python 3.7.11 (2021-06-28) fixed by commit f4dac7e (branch 3.7) (2021-05-06)
- Python 3.8.11 (2021-06-28) fixed by commit 515a7bc (branch 3.8) (2021-05-05)
- Python 3.9.5 (2021-05-03) fixed by commit 491fde0 (branch 3.9) (2021-04-29)
- Python 3.10.0 (2021-10-04) fixed by commit 76cd81d (branch 3.10) (2021-04-29)
Python issue¶
[security] urllib.parse should sanitize urls containing ASCII newline and tabs.
- Python issue: bpo-43882
- Creation date: 2021-04-18
- Reporter: Senthil Kumaran
Timeline¶
Timeline using the disclosure date 2021-04-18 as reference:
- 2021-03-16 (-33 days): Reported (email sent to the PSRT list)
- 2021-04-18: Python issue bpo-43882 reported by Senthil Kumaran
- 2021-04-29 (+11 days): commit 491fde0 (branch 3.9)
- 2021-04-29 (+11 days): commit 76cd81d (branch 3.10)
- 2021-05-03 (+15 days): Python 3.9.5 released
- 2021-05-05 (+17 days): commit 515a7bc (branch 3.8)
- 2021-05-06 (+18 days): commit 6c472d3 (branch 3.6)
- 2021-05-06 (+18 days): commit f4dac7e (branch 3.7)
- 2021-06-28 (+71 days): Python 3.6.14 released
- 2021-06-28 (+71 days): Python 3.7.11 released
- 2021-06-28 (+71 days): Python 3.8.11 released
- 2021-10-04: Python 3.10.0 released