gh-145980: Add support for alternative alphabets in the binascii module (GH-145981)

* Add the alphabet parameter in functions b2a_base64(), a2b_base64(),
  b2a_base85(), and a2b_base85().
* And a number of "*_ALPHABET" constants.
* Remove b2a_z85() and a2b_z85().

Author: serhiy-storchaka

Doc/library/binascii.rst

@@ -48,12 +48,15 @@ The :mod:`!binascii` module defines the following functions:
       Added the *backtick* parameter.
 
 
-.. function:: a2b_base64(string, /, *, strict_mode=False)
-              a2b_base64(string, /, *, strict_mode=True, ignorechars)
+.. function:: a2b_base64(string, /, *, alphabet=BASE64_ALPHABET, strict_mode=False)
+              a2b_base64(string, /, *, ignorechars, alphabet=BASE64_ALPHABET, strict_mode=True)
 
    Convert a block of base64 data back to binary and return the binary data. More
    than one line may be passed at a time.
 
+   Optional *alphabet* must be a :class:`bytes` object of length 64 which
+   specifies an alternative alphabet.
+
    If *ignorechars* is specified, it should be a :term:`bytes-like object`
    containing characters to ignore from the input when *strict_mode* is true.
    If *ignorechars* contains the pad character ``'='``,  the pad characters
@@ -76,10 +79,10 @@ The :mod:`!binascii` module defines the following functions:
       Added the *strict_mode* parameter.
 
    .. versionchanged:: 3.15
-      Added the *ignorechars* parameter.
+      Added the *alphabet* and *ignorechars* parameters.
 
 
-.. function:: b2a_base64(data, *, wrapcol=0, newline=True)
+.. function:: b2a_base64(data, *, alphabet=BASE64_ALPHABET, wrapcol=0, newline=True)
 
    Convert binary data to a line(s) of ASCII characters in base64 coding,
    as specified in :rfc:`4648`.
@@ -95,7 +98,7 @@ The :mod:`!binascii` module defines the following functions:
       Added the *newline* parameter.
 
    .. versionchanged:: 3.15
-      Added the *wrapcol* parameter.
+      Added the *alphabet* and *wrapcol* parameters.
 
 
 .. function:: a2b_ascii85(string, /, *, foldspaces=False, adobe=False, ignorechars=b"")
@@ -148,7 +151,7 @@ The :mod:`!binascii` module defines the following functions:
    .. versionadded:: 3.15
 
 
-.. function:: a2b_base85(string, /)
+.. function:: a2b_base85(string, /, *, alphabet=BASE85_ALPHABET)
 
    Convert Base85 data back to binary and return the binary data.
    More than one line may be passed at a time.
@@ -158,49 +161,25 @@ The :mod:`!binascii` module defines the following functions:
    characters). Each group encodes 32 bits of binary data in the range from
    ``0`` to ``2 ** 32 - 1``, inclusive.
 
+   Optional *alphabet* must be a :class:`bytes` object of length 85 which
+   specifies an alternative alphabet.
+
    Invalid Base85 data will raise :exc:`binascii.Error`.
 
    .. versionadded:: 3.15
 
 
-.. function:: b2a_base85(data, /, *, pad=False)
+.. function:: b2a_base85(data, /, *, alphabet=BASE85_ALPHABET, pad=False)
 
    Convert binary data to a line of ASCII characters in Base85 coding.
    The return value is the converted line.
 
-   If *pad* is true, the input is padded with ``b'\0'`` so its length is a
-   multiple of 4 bytes before encoding.
-
-   .. versionadded:: 3.15
-
-
-.. function:: a2b_z85(string, /)
-
-   Convert Z85 data back to binary and return the binary data.
-   More than one line may be passed at a time.
-
-   Valid Z85 data contains characters from the Z85 alphabet in groups
-   of five (except for the final group, which may have from two to five
-   characters). Each group encodes 32 bits of binary data in the range from
-   ``0`` to ``2 ** 32 - 1``, inclusive.
-
-   See `Z85 specification <https://rfc.zeromq.org/spec/32/>`_ for more information.
-
-   Invalid Z85 data will raise :exc:`binascii.Error`.
-
-   .. versionadded:: 3.15
-
-
-.. function:: b2a_z85(data, /, *, pad=False)
-
-   Convert binary data to a line of ASCII characters in Z85 coding.
-   The return value is the converted line.
+   Optional *alphabet* must be a :term:`bytes-like object` of length 85 which
+   specifies an alternative alphabet.
 
    If *pad* is true, the input is padded with ``b'\0'`` so its length is a
    multiple of 4 bytes before encoding.
 
-   See `Z85 specification <https://rfc.zeromq.org/spec/32/>`_ for more information.
-
    .. versionadded:: 3.15
 
 
@@ -300,6 +279,55 @@ The :mod:`!binascii` module defines the following functions:
    but may be handled by reading a little more data and trying again.
 
 
+.. data:: BASE64_ALPHABET
+
+   The Base 64 alphabet according to :rfc:`4648`.
+
+   .. versionadded:: next
+
+.. data:: URLSAFE_BASE64_ALPHABET
+
+   The "URL and filename safe" Base 64 alphabet according to :rfc:`4648`.
+
+   .. versionadded:: next
+
+.. data:: UU_ALPHABET
+
+   The uuencoding alphabet.
+
+   .. versionadded:: next
+
+.. data:: CRYPT_ALPHABET
+
+   The Base 64 alphabet used in the :manpage:`crypt(3)` routine and in the GEDCOM format.
+
+   .. versionadded:: next
+
+.. data:: BINHEX_ALPHABET
+
+   The Base 64 alphabet used in BinHex 4 (HQX) within the classic Mac OS.
+
+   .. versionadded:: next
+
+.. data:: BASE85_ALPHABET
+
+   The Base85 alphabet.
+
+   .. versionadded:: next
+
+.. data:: ASCII85_ALPHABET
+
+   The Ascii85 alphabet.
+
+   .. versionadded:: next
+
+.. data:: Z85_ALPHABET
+
+   The `Z85 <https://rfc.zeromq.org/spec/32/>`_ alphabet.
+
+   .. versionadded:: next
+
+
 .. seealso::
 
    Module :mod:`base64`

Doc/whatsnew/3.15.rst

@@ -649,13 +649,16 @@ binascii
 
   - :func:`~binascii.b2a_ascii85` and :func:`~binascii.a2b_ascii85`
   - :func:`~binascii.b2a_base85` and :func:`~binascii.a2b_base85`
-  - :func:`~binascii.b2a_z85` and :func:`~binascii.a2b_z85`
 
   (Contributed by James Seo and Serhiy Storchaka in :gh:`101178`.)
 
 * Added the *wrapcol* parameter in :func:`~binascii.b2a_base64`.
   (Contributed by Serhiy Storchaka in :gh:`143214`.)
 
+* Added the *alphabet* parameter in :func:`~binascii.b2a_base64` and
+  :func:`~binascii.a2b_base64`.
+  (Contributed by Serhiy Storchaka in :gh:`145980`.)
+
 * Added the *ignorechars* parameter in :func:`~binascii.a2b_base64`.
   (Contributed by Serhiy Storchaka in :gh:`144001`.)
 

Include/internal/pycore_global_objects_fini_generated.h

@@ -307,6 +307,7 @@ struct _Py_global_strings {
         STRUCT_FOR_ID(all)
         STRUCT_FOR_ID(all_threads)
         STRUCT_FOR_ID(allow_code)
+        STRUCT_FOR_ID(alphabet)
         STRUCT_FOR_ID(any)
         STRUCT_FOR_ID(append)
         STRUCT_FOR_ID(arg)

Include/internal/pycore_global_strings.h

@@ -56,11 +56,13 @@ def b64encode(s, altchars=None, *, wrapcol=0):
     If wrapcol is non-zero, insert a newline (b'\\n') character after at most
     every wrapcol characters.
     """
-    encoded = binascii.b2a_base64(s, wrapcol=wrapcol, newline=False)
     if altchars is not None:
-        assert len(altchars) == 2, repr(altchars)
-        return encoded.translate(bytes.maketrans(b'+/', altchars))
-    return encoded
+        if len(altchars) != 2:
+            raise ValueError(f'invalid altchars: {altchars!r}')
+        alphabet = binascii.BASE64_ALPHABET[:-2] + altchars
+        return binascii.b2a_base64(s, wrapcol=wrapcol, newline=False,
+                                   alphabet=alphabet)
+    return binascii.b2a_base64(s, wrapcol=wrapcol, newline=False)
 
 
 def b64decode(s, altchars=None, validate=_NOT_SPECIFIED, *, ignorechars=_NOT_SPECIFIED):
@@ -100,15 +102,10 @@ def b64decode(s, altchars=None, validate=_NOT_SPECIFIED, *, ignorechars=_NOT_SPE
                     break
             s = s.translate(bytes.maketrans(altchars, b'+/'))
         else:
-            trans_in = set(b'+/') - set(altchars)
-            if len(trans_in) == 2:
-                # we can't use the reqult of unordered sets here
-                trans = bytes.maketrans(altchars + b'+/', b'+/' + altchars)
-            else:
-                trans = bytes.maketrans(altchars + bytes(trans_in),
-                                        b'+/' + bytes(set(altchars) - set(b'+/')))
-            s = s.translate(trans)
-            ignorechars = ignorechars.translate(trans)
+            alphabet = binascii.BASE64_ALPHABET[:-2] + altchars
+            return binascii.a2b_base64(s, strict_mode=validate,
+                                       alphabet=alphabet,
+                                       ignorechars=ignorechars)
     if ignorechars is _NOT_SPECIFIED:
         ignorechars = b''
     result = binascii.a2b_base64(s, strict_mode=validate,
@@ -146,7 +143,6 @@ def standard_b64decode(s):
     return b64decode(s)
 
 
-_urlsafe_encode_translation = bytes.maketrans(b'+/', b'-_')
 _urlsafe_decode_translation = bytes.maketrans(b'-_', b'+/')
 
 def urlsafe_b64encode(s):
@@ -156,7 +152,8 @@ def urlsafe_b64encode(s):
     bytes object.  The alphabet uses '-' instead of '+' and '_' instead of
     '/'.
     """
-    return b64encode(s).translate(_urlsafe_encode_translation)
+    return binascii.b2a_base64(s, newline=False,
+                               alphabet=binascii.URLSAFE_BASE64_ALPHABET)
 
 def urlsafe_b64decode(s):
     """Decode bytes using the URL- and filesystem-safe Base64 alphabet.
@@ -399,14 +396,14 @@ def b85decode(b):
 
 def z85encode(s, pad=False):
     """Encode bytes-like object b in z85 format and return a bytes object."""
-    return binascii.b2a_z85(s, pad=pad)
+    return binascii.b2a_base85(s, pad=pad, alphabet=binascii.Z85_ALPHABET)
 
 def z85decode(s):
     """Decode the z85-encoded bytes-like object or ASCII string b
 
     The result is returned as a bytes object.
     """
-    return binascii.a2b_z85(s)
+    return binascii.a2b_base85(s, alphabet=binascii.Z85_ALPHABET)
 
 # Legacy interface.  This code could be cleaned up since I don't believe
 # binascii has any line length limitations.  It just doesn't seem worth it