Skip to content
69 changes: 37 additions & 32 deletions pep-0675.rst
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,13 @@ accept arbitrary literal string types such as ``Literal["foo"]`` or
Motivation
==========

A common security vulnerability is for a program to include
user-controlled data in a command it executes. For example, a naive
way to look up a user record from a database is to accept a user id
and insert it into a predefined SQL query:
Powerful APIs that execute SQL or shell commands often recommend that
they be invoked with literal strings, rather than arbitrary user
controlled strings. There is no way to express this recommendation in
the type system, however, meaning security vulnerabilities sometimes
occur when developers fail to follow it. For example, a naive way to
look up a user record from a database is to accept a user id and
insert it into a predefined SQL query:

::

Expand Down Expand Up @@ -157,25 +160,28 @@ example:

Notice that the user did not have to change their SQL code at all. The
type checker was able to infer the literal string type and complain
only in case of violations. The ``Literal[str]`` type is also useful
in other cases where we want strict command-data separation, such as
when building shell commands or when rendering a string into an HTML
response without escaping (eg. via Django's ``mark_safe``
function). Overall, this combination of strictness and flexibility
makes it easy to enforce safer API usage in sensitive code without
burdening users.
only in case of violations.

``Literal[str]`` is also useful in other cases where we want strict
command-data separation, such as when building shell commands or when
rendering a string into an HTML response without escaping (see
`Appendix A: Other Uses`_). Overall, this combination of strictness
and flexibility makes it easy to enforce safer API usage in sensitive
code without burdening users.

Usage statistics
----------------

In a sample of open-source projects using ``sqlite3``, we found that
``conn.execute`` was called `~67%
``conn.execute`` was called `~67% of the time
<https://grep.app/search?q=conn%5C.execute%5C%28%5Cs%2A%5B%27%22%5D&regexp=true&filter[lang][0]=Python>`_
of the time with a safe string literal and `~33%
with a safe string literal and `~33% of the time
<https://grep.app/search?current=3&q=conn%5C.execute%5C%28%5Ba-zA-Z_%5D%2B%5C%29&regexp=true&filter[lang][0]=Python>`_
of the time with an unsafe, dynamically-built local variable. Using
this PEP's literal string type along with a type checker would have
prevented ``execute`` from being called in such an unsafe manner.
with a potentially unsafe, local string variable. Using this PEP's
literal string type along with a type checker would prevent the unsafe
portion of that 33% of cases (ie. the ones where user controlled data
is incorporated into the query), while seamlessly allowing the safe
ones to remain.

Rationale
=========
Expand Down Expand Up @@ -223,13 +229,13 @@ string?
We want to specify that the value must be of some type
``Literal[<...>]`` where ``<...>`` is some string. This is what
``Literal[str]`` represents. ``Literal[str]`` is the "supertype" of
all literal string types. Any particular literal string such as
``Literal["foo"]`` or ``Literal["bar"]`` is compatible with
``Literal[str]``, but not the other way around. The "supertype" of
``Literal[str]`` itself is ``str``. So, ``Literal[str]`` itself is
compatible with ``str``, but not the other way around. In effect, this
PEP just introduces a type in the type hierarchy between
``Literal["foo"]`` and ``str``.
all literal string types. In effect, this PEP just introduces a type
in the type hierarchy between ``Literal["foo"]`` and ``str``. Any
particular literal string such as ``Literal["foo"]`` or
``Literal["bar"]`` is compatible with ``Literal[str]``, but not the
other way around. The "supertype" of ``Literal[str]`` itself is
``str``. So, ``Literal[str]`` is compatible with ``str``, but not the
other way around.

Note that a ``Union`` of literal types is naturally compatible with
``Literal[str]`` because each element of the ``Union`` is individually
Expand Down Expand Up @@ -523,8 +529,13 @@ match:
s: str
x3: str = foo(s) # Third overload.


Backwards Compatibility
-----------------------
=======================

``Literal[str]`` is acceptable at runtime, so
this doesn't require any changes to the Python runtime itself. :pep:`586`
already backports ``Literal``, so this PEP does not need to change it.

As :pep:`PEP 586 mentions
<586#backwards-compatibility>`,
Expand Down Expand Up @@ -553,18 +564,12 @@ annotating it as ``x: Literal[str]``:
x: Literal[str] = "hello"
expect_literal_str(x)

Runtime behavior

Runtime Behavior
================

This PEP does not change the runtime behavior of ``Literal``.

Backwards compatibility
=======================

Backwards compatibility: ``Literal[str]`` is acceptable at runtime, so
this doesn't require any changes to the Python runtime itself. :pep:`586`
already backports ``Literal``, so this PEP does not need to change it.


Rejected Alternatives
=====================
Expand Down