Avoid a theoretical integer overflow in base64_encode()

If base64_encode() were to be used with a huge data array, the
previous version could have resulted in overwriting the allocated
buffer due to an integer overflow as pointed out in
http://www.freebsd.org/cgi/query-pr.cgi?pr=137484. However, there
are no know use cases in hostapd or wpa_supplicant that would do that.
Anyway, the recommended change looks reasonable and provides additional
protection should the base64_encode() function be used for something
else in the future.

diff --git a/src/utils/base64.c b/src/utils/base64.c
index 0eadb81..13fc511 100644 (file)
--- a/src/utils/base64.c
+++ b/src/utils/base64.c
@@ -43,6 +43,8 @@ unsigned char * base64_encode(const unsigned char *src, size_t len,
        olen = len * 4 / 3 + 4; /* 3-byte blocks to 4-byte */
        olen += olen / 72; /* line feeds */
        olen++; /* nul termination */
+       if (olen < len)
+               return NULL; /* integer overflow */
        out = os_malloc(olen);
        if (out == NULL)
                return NULL;