A POSIX Env-Game

So RC2014WW didn’t go so well… needing to provide a very few POSIX.1-2001 capabilities on Solaris 2.6, I got a bit distracted, aiming for a perfect implementation.

The problem is the multitude of mutually-incompatible-by-design de-jure standardised APIs for manipulating environment-variables:

  • Direct access/manipulation of the global variable char **environ; and of manipulating the pointed-to pointers-to-strings, and the strings themselves.
  • Direct access/manipulation of the third argument to main(), in the same manners as above.
  • putenv(), the non-allocating environment-modifier inherited from the XPG4 standard; being non-allocating, you can directly alter the name and contents of the environment-variable after having set it – without calling any API function (ouch!) – by simply updating the pointed-to characters, eg:
    static char junk[128];  /* must be static! */
    strcpy(junk, "BIGGLES=scarf");
    putenv(junk);    /* env: BIGGLES=scarf */
    junk[2] = junk[3] = 'B';  /* now: BIBBLES=scarf */
    strcpy(&junk[9], "tupid");  /* now: BIBBLES=stupid */

    which is fairly braindamaged, and is explicitly allowed (described and documented) in the IEEE Std 1003.1-2001 API Specification (POSIX 1003.1-2001).

  • setenv()/unsetenv(), the more modern allocating/deallocating equivalents to putenv().

Because putenv() does not allocate, but can be used to delete or update variables allocated and set by setenv(), it doesn’t take a rocket-scientist to see that there be potential memory-leaks lurking there that are completely outside the control of the API-user, not to mention invalid deallocations when used the other way around.

Therefore putenv() needs to know about the internal implementation details of setenv() and unsetenv(), and vice versa. Worse, the internal implementation details concerned have to be explicitly constructed in such a way as to make compatibility between putenv() and setenv() even possible.

Thus, if your environment has putenv(), but not setenv(), you cannot just craft-up a setenv() function to fill the gap – you have to replace the existing putenv() as well. And sometimes, depending on the exact internal implementation of setenv() (which is often not available for inspection), the same applies the other way around, too.

Mixing Global Variables and API Functions – A Bad Idea
Although the structure of environment variable storage are exactly defined by POSIX, their storage-class is not – it is treated as an implementation characteristic. In other words, at program startup, the environment-variable array and pointed-to strings could be stored on the heap, the stack, the static data-section, in alternate-bank memory, or even in physically read-only memory. Of course, only one of those can be deallocated, so a portable implementation of unsetenv() cannot use free() on any of the initial environment strings, or on the global array-pointer either.

This in turn means that the first invocation of any one of the putenv(), setenv() or unsetenv() functions must transfer the strings and the global array of pointers into allocated heap memory, and update the global “environ” pointer, to be able to safely proceed.

Unfortunately, that update of the global array-pointer immediately invalidates the third argument to main() – thus the POSIX.1-2001 standard explicly notes that accessing the environment via the third argument to main() after any call to getenv()/putenv()/setenv()/unsetenv() will result in unspecified behaviour… possibly even a segmentation violation or arbitrary memory-corruption.

That’s useful…NOT!

putenv() Argument Validation – Cannot Be Bullet-Proof
As can be imagined, a putenv() implementation should ensure that the passed-in string is validated – it must contain an equals-sign. Unfortunately, because the programmer still has direct-access to the putenv-ed string, they can still corrupt the environment storage without any deviousness at all:

static char str[128];
strcpy(e, "LC_ALL,C");
if (putenv(e) < 0)  /* no equals sign, but catchable */
strcpy(e, "LC_ALL=C");
if (putenv(e) < 0)          /* OK so far... */
e[6] = '+';  /* we have now corrupted the storage 
              * by overwriting the equals sign... */

It is precisely for these reasons that setenv() takes separate arguments for the Name and Value of the environment-variable concered, and also why it always allocates the storage necessary for the resulting environment-variable string.

Finagling the Root Pointer
Of course, even if putenv() did not exist, the programmer could provoke similar problems by directly manipulating the pointed-to strings or the pointers via the global environ pointer – or even by changing the global environ pointer to point at a completely different array of pointers-to-strings. Jeepers!

static char *faked[] = {
      "ALPHA=first", "BETA=second", "GAMMA=third", NULL
environ = faked;
setenv("ALPHA", "newval", 1);  /* must not try to deallocate
                                 the old value!!! */

To be able to catch this situation (and free the storage previously allocated via the old root pointer, to prevent a memory-leak), putenv(), setenv() and unsetenv() need to “remember” the previous value of the global root environ pointer, and if it changes, do the necessary cleanup. Of course, the program concerned could later change it back, but fortunately for implementors, that is expressly described as invoking “undefined behaviour”, ie: the program cannot rely on anything if it reverts the environ pointer after calling any of putenv(), setenv() or unsetenv().

Aside: An API-Users Perspective
Given the strangeness from mixing the different formally-standadised access/update mechanisms, one would expect that the programmers rule-of-thumb would be:

  1. If you have setenv()/unsetenv(), use them exclusively: do not call putenv(), and do not manipulate directly via the global env pointer or the third argument to main().
  2. If you do not have setenv()/unsetenv(), use putenv() but with great care: never update the characters pointed-to by the argument you passed to putenv(); and make sure that the pointer you pass to it points to storage with either static or allocated lifetime (ie: not a pointer to any “auto” or non-static local variables).
  3. If you have neither putenv nor setenv(), write your own local setenv()/unsetenv() functions, as per the POSIX.1-2001 specification.
  4. In no case ever update the global environ pointer or any of the string-pointers in the array.

So what does Netrek server do? It uses both putenv() and setenv(). Not good – it means that I will have to provide the full leak-proof, pointer-tracking, shared-knowledge-implementation of putenv(), setenv() and unsetenv()… as described in the next article, POSIX *env API: unleaking portable version.