Category: Code

Posts related to programming.

find: files only with scm directory pruning

The version of find I’m discussing here is
find (GNU findutils) 4.7.0-git
I use this pattern frequently—
$ find . <conditions> |xargs grep <pattern>
to find files containing, say, a regular expression.  If the search tree contains mercurial or git directories, I usually want to exclude their contents from the search.

The -prune action prevents a search from descending into the pruned directory, but I also want to strip out all directories, because the filenames are being fed into xargs grep.  So the command feeding the xgrep looks something like—
$ find . -type d -name .hg -prune -o -type f -print
This works well.  All directory names are suppressed, along with the files contained in the .hg directory.

Because the default action on a find is -print, I often elide that action, so I end up with—
$ find . -type d -name .hg -prune -o -type f
Lo and behold, the name of the pruned .hg directory appears in the list of files passed to xargs.  All other directory names are suppressed.

What seems to be going on is this: the condition before the -o finds only directories named .hg.  Those it prunes, but the condition returns the names of the pruned directories.  The condition following the -o filters out all of the directories not named .hg.  The combined list of files and .hg directories (but not their contents) is passed to xargs.

So how is it that the first version works as I want it to?  How are the names of the .hg directories suppressed?

All I can surmise is that, in the absence of a specific -print action, the default -print applies to each of the conditions, but when it is specifically applied to the -o conditions, the default is suppressed for the initial conditions.

The man page says:
If the whole expression contains no actions other than -prune or -print, -print is performed on all files for which the whole expression is true.
That is ambiguous; in fact, it seems to be false. Neither of the versions above contain any actions apart from -prune and -print, and one doesn’t even contain a -print.  Yet they behave differently.

The command
$ find . \( -type d -name .hg -prune -o -type f \) -print
which pops the -print out from within the sub-expression, behaves the same as
$ find . -type d -name .hg -prune -o -type f
so that seems to be what is effectively happening in the absence of a specific -print on the or condition. (Incidentally, you seem to be able to use -or in place of just -o.)

tikadiff: graphical diff for text from “binary” files

Code

The code is from the Downloads area of my Atlassian Bitbucket repository; see the README online.

Version Control Systems (VCSs)

VCSs like mercurial, git and bazaar (to mention only a few) are great for keeping track of changes to source files, but their utility doesn’t stop there.  If you’re working on documents in applications like Word, OpenOffice or LibreOffice, especially when you are asking others to review those documents, a VCS program can save you a lot of anguish.

However, people who work not with source code, but with research papers, academic assignments and the like, are not inclined to make themselves familiar with the tools that geeks have grown used to.  Considering how long it took for software developers to embrace those tools, it’s hardly surprising.

Limitations of VCSs

Unfortunately, VCSs aren’t set up well for managing such files.  They depend for efficient version management on tracking line-by-line differences in file.  This allows them to maintain the minimal set of changes from version to version, and to readily show what changes have occurred between any two versions.  That works well for source files, which are written in plain text, but not for files which maintain their own complex internal formats, which generally only become readable when translated by the “mother” application.

VCSs provide for such files, but they mark them as binary, and make no attempt to track the differences between them.  Instead, for each version, they simply keep a copy of the entire file.  While this is still better by a long stretch than not being able to track, version by version, the history of a document, it makes it impossible for the native VCS to show what the differences between versions actually were.

Other solutions

This problem has been partially addressed before. For those who generate Open Document Text (ODT) files with, for example,  OpenOffice, the program odt2txt will extract text from those documents.  odt2txt, in turn, enabled oodiff, which is a script to generate a diff from the text of two ODT files, extracted with odt2txt.  It’s not a complete solution, because there is still no merge facility, but it at least allows the textual differences between versions to be examined from within the VCS.

The combination of such a file comparison program with a VCS graphical interface presents a much lower barrier to adoption for users from outside the asylum.  Because I᾽m on OS X,  I’ve been using Atlassian’s free product, SourceTree, which has the advantage of working with both mercurial (hg) and git.  If you’re working with mercurial only, or if you are on a linux distribution, you can use TortoiseHg as a graphical front end.

That wasn’t quite enough, because my wife, whose requirements got me thinking about this, works mainly with Word files.  I needed an equivalent of odt2txt and oodiff for both .doc and .docx files — which have very different formats.

Thank you, Tika

Fortunately, the problem of extracting text from these formats has already been conveniently solved by the Apache Tika project.  Tika can extract metadata, plain text, xml and html from a dazzlingly array of file types.  For my purposes, plain text and metadata will suffice; metadata only for any file types, like images, from which text cannot be gleaned.  Tika provides not merely a substitute for odt2txt, but an extension to virtually any commonly (an many not so commonly) used file formats. All that remained was to provide a substitute for oodiff.

tikadiff

That’s tikadiff.  It takes either two filenames and generates a graphical diff of them, or two directories and, for each file in the first, generates a diff of the file of the same name (if it exists) in the second.  tikadiff depends on Tika (obviously) and a graphical diff program; by default kdiff3, but it is currently written to look also for Perforce p4merge, and can be instructed to use a diff program of your choice, provided that it accepts the same two arguments.

tikadiff depends also on a number of scripts that are distributed with it.

tika

tika is a convenience script to run the CLI from the tika-app jar file.  It passes all its arguments to tika-app.  In addition to that basic function, it will preset certain arguments to tika-app depending on the name by which it is invoked.

  • tikatype —prints the mimetype of the file named in its argument
  • tikameta —prints the metadata of the file named in its argument
  • tikatext —prints the plain text extracted from the file named in its argument
  • tikaxml —prints the XML extracted from the file named in its argument
  • tikahtml —prints the HTML extracted from the file named in its argument

tikserve

tikserve, like tika, is a convenience script to run tika-app.  Unlike tika, it is not invoked (except during setup) under its own name, but only through a series of links.  It runs tika-app as a server, which performs some operation on any file which is written to its open TCP port.

  • tikstype —set up server to return mimetype of file
  • tiksmeta —set up a server to return the metadata of the file
  • tikstext —set up a server to return the plain text extracted from the file
  • tiksxml —set up a server to return the XML extracted from the file
  • tikshtml —set up a server to return the HTML extracted from the file
tikadiff only uses tikstype, tiksmeta and tikstext when comparing directories, on the theory that it will be faster to use a server when testing and comparing multiple files.  I have not checked whether this theory is valid.

In order to run the server(s), tikserve needs a port on localhost. To support this habit, tikserve looks to two other scripts.

freeport

freeport hands back the next available port on localhost, starting at 1024. You may optionally give it a minimum port number and, sub-optionally, a maximum port number as constraints.  In order to perform this task, freeport requires—

localhostports

localhostports prints the ports which, in the opinion of netstat, are currently associated with localhost.

Code, again

Bitbucket repository Downloads; see the README online.

Help for digest checking

Updated 2015-04-19

It’s pretty important to check the digests of software you download.  When a downloaded file is accompanied by a signature file, for example a gnupg .asc file, you can verify the signature with various tools.  Often though, a download site will include the MD5 or SHA1 digest hash of the file, which allows a quick check on the file’s integrity.  OS X has an /sbin/md5 command, and includes the openssl distribution.  Within openssl, the digest subcommand allows for the generation of digests for an array of digest algorithms, including MD5 and SHA1.  So it’s simple enough to generate the appropriate digest for that file you just downloaded.

Comparing them is a bit tedious, though.  If you’re like me, you skim across the two digests – the one you generated and the one that the authors published – and look for eye-catching patterns near the beginning, middle and end.  That works pretty well in practice, but its hardly rigorous.

So I wrote myself a little script that generates a hash, and checks it against a hash value given as an argument.  Here’s the help output.
 $ sha1 --help
Usage: sha1 &lt;filename&gt; [test-value]
where &lt;filename&gt; is required and test-value is
an optional hex digest value to test against.
Exits with
1 if the number of arguments is wrong;
2 if the digest comparison fails;
3 if an unsupported algorithm is requested.

or: sha1 --digests
which will echo the supported algorithms.

or: sha1 --setup
which tries to ensure that hard links are set up for all digest algorithms.
Exits with 4 if NOT all links are available after execution.

Note that the user running this option must have write permission in the
directory containing the invoked program. Soft links are followed
to determine the actual directory containing the program.
Alternatively, you may manually set up your own softlinks in any directory.

or: sha1 -&lt;anything&gt;
Print this help text.

Some of the executables may already exist on your system. On OS X, for example,
the executable /sbin/md5 is present. Which one is found depends on your PATH.
$

Installing the script

Download the script from here.  The SHA1 digest of the script is
59254e751d0ce827770a73aae573f5294a1e1ac9
This assumes that your downloader is happy with UTF-8 or ASCII and the existing Unix line endings (LF). If it changes them, all bets are off.

Select a location on your PATH.  I installed in ~/bin, but you may want to put it in /usr/local/bin or some such system directory.  To do that you will have to use sudo or some equivalent.  Make sure the file is executable.  In my case, I would do this:
$ mv ~/Downloads/sha1 ~/bin/sha1
$ chmod +x ~/bin/sha1

Set up links

The –setup option will try to set up hard links for each of the supported digest algorithms (a subset of the algorithms supported in openssl digest).  If you required sudo to install the script initially, you will also have to run the –setup as sudo. Your output will look something like this.
$ sha1 --setup
Executable /Users/pbw/bin/sha1 already exists.
Executable /Users/pbw/bin/md5 created.
Executable /Users/pbw/bin/sha256 created.
Executable /Users/pbw/bin/sha384 created.
Executable /Users/pbw/bin/sha512 created.
Executable /Users/pbw/bin/dss1 created.
$
Now that it’s installed you can check a digest like so.  I’ll go to the Downloads folder and check (too late now, of course) the initial download.
$ sha1 sha1 59254e751d0ce827770a73aae573f5294a1e1ac9
sha1 OK
59254e751d0ce827770a73aae573f5294a1e1ac9 EQUALS
59254e751d0ce827770a73aae573f5294a1e1ac9
$
Let’s see what happens if you change the comparison text.
$ sha1 sha1 59254e751d0ce827770a73aae573f5294a1e1acA
sha1 FAIL
59254e751d0ce827770a73aae573f5294a1e1ac9 NOT EQUAL TO
59254e751d0ce827770a73aae573f5294a1e1aca
$

The script returns true (0) for a successful comparison and false (in this case, 2) for a failure.

That’s it folks!

Monad; that’s a wrap!

Just like everybody else who starts to look at monads, I found it was like coming to the face of a sheer cliff.  Let me qualify that: just like every other programmer who is not a mathematician (and that’s most of us).  I am looking at monads in the context of clojure, so code snippets will generally be written in clojure-like syntax.

I have found these resources the most helpful:

Adam’s talk eschews category theory, and mathematics in general, concentrating on how to define and use monads in code.  This is an essential approach for the rest of us. Unfortunately, all of the terms used in discussing monads hark back to the mathematics, and that, I believe, makes the practical use of monads more confusing than it need be.  This point was brought home to me strongly when, after observing what some of the others were saying, I watched the first part of a lecture by this extraordinary woman.

Warning: take everything I say with a large grain of salt.  I am writing this to help me to sort it out in my own mind, and there are no guarantees that any of it is correct.

In all of the discussions I have seen, it is stressed that monads are defined in terms of the behaviour of two functions.

wrap

Although it is not usually mentioned first, I will start with the function that I will call wrap.  Nobody else calls it that, but that’s what it does. It goes by a number of aliases:

  • result, m_result, mResult, etc.
  • return, m_return, etc.
  • lift, m_lift, etc.
  • the monadic function (? Sometimes. However, the 2nd argument to rewrap —see below— is generally called monadic function, so it may be more accurate to describe wrap as the base function on which all other monadic functions of a given monad are built.)
When he discusses the Array Monad for Javascript, Santosh Rajan gives the following signature for the monadic function, i.e. wrap.

f: T -> [T]

That is, for the Array Monad, wrap takes a value of some type T, and wraps it in an array.  Both the type of value and the mode of wrapping are specific to each defined monad, but once defined, they are fixed for that monad.

The resulting, wrapped, value returned from wrap is known as a monadic value, and is often represented in code as mv.  In this discussion, I’ll call it wrapt, for obvious reasons. I’ll call the argument to wrap the basic value, or bv.

;; Takes a basic value; returns a wrapt value
(defn mymonad-wrap [bv]
  (let [wrapt (do-the-wrapping-of bv)]
    wrapt))

You can see why monadic values are frequently called containers.

(unwrap)

unwrap is under wraps, so to speak, because it is not part of the public face of monads. It is NOT one of the two functions which define the behaviour of a monad.  It is, however, essential to the functioning of monads, and some kind of unwrap functionality must be available to the other function in the definition of monad: rewrap.

unwrap is the inverse of wrap, not surprisingly.  In terms of Santosh’s Array monad, it’s signature might look like this.

f: T <- [T]

That is, unwrap takes a wrapt (monadic value), which is a wrapped basic value and returns the basic value.

rewrap

This the function that is generally called bind, m_bind, etc., although in Santosh’s examples, this function takes the name of the monad; for example, arrayMonad.  The signature that Santosh gives for this function in the Array Monad is

M: [T] -> [T]

That is, it transforms a wrapt value to another wrapt value. In the case of the Array monad, the basic value is wrapped in an array.

rewrap looks like this.

(defn mymonad-rewrap [wrapt, mf]
  ;; call monadic function given as 2nd arg, on the
  ;; basic value extracted from the wrapt value given
  ;; in the 1st argument. Return the new wrapt value.
  (let [bv (do-unwrap-fn-on wrapt)
        new-wrapt (mf bv)]
    new-wrapt))

So, what rewrap does is

  • unwrap its monadic value argument to get the basic value
  • call its monadic function argument with the unwrapped basic value to…
    • modify the basic value
    • wrap the modified basic value and return a new wrapt value

The monadic function argument, mf, deserves a closer look.  mf operates on a basic value to produce a new wrapt value.  It is, in fact, a composition of functions.  It composes wrap and some operation that modifies the basic value.  So,

mf ⇒ (f′ ⋅ wrap)

where f′ is a function that modifies the basic value. In that scheme, wrap itself can be described as

wrap′ ⇒ (identity ⋅ wrap)

That given, we can now describe rewrap as

(defn rewrap [wrapt, (f′ ⋅ wrap)]
  (let [bv (unwrap wrapt)
        new-wrapt (wrap (f′ bv))]
    new-wrapt))

or, equivalently,

(defn rewrap [wrapt, (f′ ⋅ wrap)]
  (wrap (f′ (unwrap wrapt))))

The 3 R’s

Monads must obey three rules.  These rules I have taken from Jim Duey’s post, with the appropriate translation to the “wrap” terminology I’m using here.

Rule 1

(rewrap (wrap x) f) ≡ (f x)

Alternatively, given our rewriting of rewrap, above, but using f rather than (f′ ⋅ wrap);

(f (unwrap (wrap x)) ≡ (f x)
⇒ (f x) ≡ (f x)

Rule 2

(rewrap wrapt wrap) ≡ wrapt
⇒ (wrap (unwrap wrapt)) ≡ wrapt
⇒ wrapt ≡ wrapt

Rule 3

(rewrap (rewrap wrapt f) g) ≡
    (rewrap wrapt (fn [x] (rewrap (f x) g)))
LHS ⇒ (rewrap (f (unwrap wrapt)) g)
    ⇒ (g (unwrap (f (unwrap wrapt))))
    ⇒ (g (unwrap (f x)))                 [1]
    ⇒ (g (unwrap (wrap (f′ x))))
    ⇒ (g (f′ x))
RHS ⇒ (rewrap wrapt (fn [x] (g (unwrap (f x)))))
    ⇒ ((fn [x] (g (unwrap (f x)))) (unwrap wrapt))
    ⇒ ((fn [x] (g (unwrap (f x)))) x)    [2]

Everything looks pretty straightforward, except for the correspondence between [1] and [2] in Rule 3. Something seems odd about it, even though the results will be the same.

zargrep: grep files in a zip archive

How do you search for strings within a zip archive?

I’m tinkering with EPUB3 files, and I wanted to be able to find certain strings within .epub files, so I had a look around, and I immediately found zgrep and family. The trouble was that zgrep assumes a single zipped file, not an archive.

So, without further ado, I wrote the following script, which I called, naturally, zipgrep. It uses grep and unzip, which it assumes to be available on the PATH.  Not wanting to have to pick through the argument list, I decided to mark the end of arguments to grep with the traditional ‘—‘, after which I could stack up as many zip file names as I liked.

It was a case of not enough time in the library; or Google, in this case.  As soon as I had it working, I discovered the original zipgrep.

All was not lost.  The original zipgrep handles a single archive using egrep and unzip, with the nice wrinkle of optional sets of filenames to include in, or exclude from, the search.  However, I liked the ability to search multiple zip archives, and grep can be converted to any of its relatives with an appropriate flag, so I decided to hang on to son of zipgrep.  All I needed was a new name: hence zargrep.

You can retrieve it here. It has been tested on OS X against multiple EPUB3 files.

Because they are zip files, this should also work for jar files, but I haven’t yet tried it.

 #! /bin/sh  
   
 # Greps files in a zip archive.  
 # Same argument sequence as for grep, except that  
 # zip file arguments must be separated from flags and  
 # patterns by --. If no -- is found in the argument list, returns error.  
   
 usage() {  
   echo Usage: >&2  
   echo $0 "<grep flags> <pattern> -- zipfiles ..." >&2  
 }  
   
 declare -a args  
   
 i=0  
 for (( i=0; $# > 0; i++ ))  
 do  
   if [ "$1" != "--" ]; then  
     args[$i]="$1"  
     shift  
   else  
     filesmarked=1  
     shift  
     break  
   fi  
 done  
   
 if [ -z "$filesmarked" ]; then  
   Echo "No '--' marker for zipfiles args." >&2  
   usage  
   exit 1  
 fi  
   
 tmpfile=/tmp/zipgrep$$  
 rm -rf $tmpfile  
 mkdir $tmpfile  
   
 trap 'rm -rf $tmpfile' EXIT  
   
 wd=$(pwd)  
 cd $tmpfile  
   
 while [ $# -gt 0 ]; do  
   zipfile="$1"  
   zfile="$1"  
   shift  
   # If zipfile is not absolute, set it relative to wd  
   if [ "${zipfile:0:1}" != / ]; then  
     zipfile="$wd/${zipfile}"  
   fi  
   unzip "$zipfile" >/dev/null  
   result=$(find . -type f -print0|xargs -0 grep "${args[@]}")  
   if [ -n "$result" ]; then  
     echo "zip: $zfile"  
     echo "$result"  
   fi  
   cd $wd  
   rm -rf $tmpfile  
   mkdir $tmpfile  
   cd $tmpfile  
 done

Setting environment variables in OS X Sierra, El Capitan and Yosemite

UPDATE for Sierra and El Capitan

The changes between Mavericks and Yosemite, noted below, still apply in Sierra. At least, I have not had to change anything since Yosemite for either El Capitan or Sierra.

UPDATE for Yosemite.

Apple and Tim Cook have shown their commitment to the wider developer community, once again, by starting to pull the plug on the launchctl facilities for setting and modifying environment variables.  Here’s the rub: with the last lot of changes, Apple got the system working very, very well.

You can get a feel for the impact of the changes by reading the launchctl man page.  See, in particular, the section LEGACY SUBCOMMANDS.  It’s not entirely accurate, but that’s not unusual.  The critical subcommands are getenv, setenv, and unsetenv. The man page indicates that the export subcommand is available; it is not. (See update in the text, below.)

As far as I know, the procedure outlined below is still valid for Yosemite, but the question is still open.  I would not be surprised to learn that the behaviour of launchctl has changed in ways which invalidate some of this discussion.  If you find any problems, please let me know in comments.

In a previous post I explained how to integrate your .profile settings with the desktop’s background environment on Lion, using ~/.MacOSX/environment.plist and code in .profile.

Then, of course, Apple changed all that when they released Mountain Lion. To achieve the same objective now required launchctl, and environment.plist was orphaned. Not wanting to abandon the integration path from .profile settings to the desktop environment, I built a method for using launchctl on top of my existing environment.plist code. That got pretty arcane, but it did the job for me.

There was always the problem with the first method, that in order for environment.plist to “take,” you had to go through a login cycle. launchctl overcame that problem. In fact, launchctl was a great advance, because it enabled immediate update of the desktop environment. I’ve now rewritten the .profile code and the launchctl setup to  completely remove all reference to the plist. In the process, launchctl provides immediate updates of the desktop environment variables, and they persist across login and reboot cycles.

As explained in a previous post, the now-recommended method of system variable setting is to use the setenv subcommand of launchctl, like so:

launchctl setenv M2 /usr/share/maven/bin

The strings defined in setenv statements like the above, must be string literals. There is no means of resolving variable content based on other variables. For example, you cannot define
setenv TMPDIR $HOME/tmp
You can check the state of the launchd environment with the subcommand export:
launchctl export

This will output a series of shell commands of the form

M2=" /usr/share/maven/bin"; export M2;
Nope, not any more.  Tim Cook doesn’t think you need this.  Thanks Tim.  I’m looking forward to finding out what else I don’t need.

Here’s the modified code in .profile to set both launchctl and the bash environment variables.
Create the following functions in .profile.
export LAUNCHCTL_ENV_VARS="$HOME/.launchctl_env_vars"
if [ -f $LAUNCHCTL_ENV_VARS ] ; then rm $LAUNCHCTL_ENV_VARS; fi

set_env_var () {
eval export $1=\""$2"\"
set_launchctl_env "$1" "$2"
}
unset_env_var () {
unset $1
unset_launchctl_env "$1"
}
set_launchctl_env () {
eval launchctl setenv "$1" \""$2"\"
echo launchctl setenv "$1" \""$2"\" >>$LAUNCHCTL_ENV_VARS
}
unset_launchctl_env () {
eval launchctl unsetenv "$1"
echo launchctl unsetenv "$1" >>$LAUNCHCTL_ENV_VARS
}

You may then use the function set_env_var to set both bash and launchctl entries. For example,
set_env_var M2_HOME "/usr/share/maven"
set_env_var M2 "$M2_HOME/bin"
set_env_var HTML_TIDY "$HOME/.tidy"

Note that the environment variables will immediately be available to any shell scripts or Terminal invocations, and to any applications launched from the desktop.

To get the environment re-established across a login, the launchctl startup features have to be invoked. This is where the file ~/.launchctl_env_vars comes in.  Notice that in the .profile code above, we have been executing a launchctl setenv command, and immediately echoing that same command to ~/.launchctl_env_vars.

When the system starts, the launchd process finds system daemon processes to launch from /System/Library/LaunchDaemons & /Library/LaunchDaemons. When a user logs in, launchd looks for user agents to start up in /System/Library/LaunchAgents, /Library/LaunchAgents & ~user/Library/LaunchAgents. While the first two are system-owned directories, the third is owned by the individual user.

We need two files: 1) a shell script which will actually issue the launchctl setenv commands, and 2) a LaunchAgent file that will tell launchd where to find the executable, and how to run it. The shell script is available here; the LaunchAgent file is available here.

The executable: profile2launchctl

#!/bin/sh
#
cmd_list="$HOME/.launchctl_env_vars"
SLEEP_TIME=10
# Uncomment following to echo launchctl commands to stdout
# Key StandardOutPath will have to be set in the plist file
# (au.id.pbw.plist2launchctl) for output to be captured.
#ECHO_TO_STDOUT=true
#
one_cmd () {
eval "$@"
}
#
[[ -n "$ECHO_TO_STDOUT" ]] && cat "$cmd_list"
cat "$cmd_list" | while read line; do one_cmd $line; done
[[ -n "$ECHO_TO_STDOUT" ]] && echo Sleeping in profile2launchctl
#
# Sleep for a while so launchd doesn't get upset
[[ -n "$SLEEP_TIME" ]] && sleep "$SLEEP_TIME"
Notes:
The required sequence of launchctl setenv is in the cmd_list file. The commands are simply read one line at a time,  and each line is handed to eval for execution.
Permissions & Location:
To stay on the safe side, give the file -rwxr-xr-x permissions. It should be placed in /usr/libexec, which is root owned. You will have to use sudo to copy it.

The LaunchAgent file: au.id.pbw.profile2launchctl

<? xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>au.id.pbw.profile2launchctl</string>
<key>KeepAlive</key>
<false/>
<key>Program</key>
<string>/usr/libexec/profile2launchctl</string>
<key>RunAtLoad</key>
<true/>
<key>UserName<key>
<string>pbw</string>
</dict>
</plist>

Notes:
The UserName field will, of course, be set to your own login name.  Likewise, the au.id.pbw can be replaced with any suitable identifier, and the log file directed to a suitable location.
If you need to debug the plist file, add the following lines.

<key>StandardOutPath</key>
<string>/tmp/plist.log</string>
<key>Debug</key>
<true/>

These lines are used in conjunction with the ECHO_TO_STDOUT variable in the executable file. If used, the StandardOutPath will point to a writable file on your system. In this case, it points to a file in my home directory; you will have to change that.
Permissions & Location:
Again, give the file -rwxr-xr-x permissions. Place it in your $HOME/Library/LaunchAgents directory.

 

Bash functions for path manipulation

Here is a set of path manipulation functions that I first wrote in 2000, and have tinkered with on and off since then.

The functions:

  • Path stripping functions
    • rem_path
      • Removes the individual path argument(s) from PATH.  Implemented as rem_vpath PATH args…
    • rem_vpath
      • Removes the individual path argument(s) from the path specified by the first argument; e.g. rem_vpath CLASSPATH /some/class/path.jar …
      • chk_delim
        • A subsidiary function of rem_vpath which tries to find a safe regular expression delimiter for the sed removal code.
    • clean_path
      • Removes second and subsequent duplicate entries from PATH. Implemented as clean_vpath PATH
    • clean_vpath
      • Removes second and subsequent duplicate entries from the path named in the first (and only) argument; e.g clean_vpath CLASSPATH
  • Path building functions
    • app_path
      • Appends the argument(s) to PATH. Any duplicates are then removed. Implemented as app_vpath PATH args…
    • app_vpath
      • Prepends the second and subsequent arguments(s) to the path named in the first argument.  Any duplicates are then removed.  
        E.g.  app_vpath CLASSPATH /some/class/path.jar …
    • pre_path
      • Prepends the argument(s) to PATH. The arguments are added in order of arguments. Any duplicates are then removed. Implemented as pre_vpath PATH args…
    • pre_vpath
      • Prepends the second and subsequent arguments(s) to the path named in the first argument. The paths appear in order of arguments. Any duplicates are then removed.  
        E.g.          pre_vpath CLASSPATH /some/class/path.jar …
  • Path listing functions
    • list_path
      • Lists the elements of path, one per line. Implemented as list_vpath PATH
    • list_vpath
      • Lists the elements of the path named in the first (and only) argument, one per line.

The file is available online.

# N.B  The following PATH manipulation functions
# 1) handle the empty path "", but do not equate the empty path with "."
# 2) will fail on all rem_* functions if
#    ALL of the characters '|' '%' ',' '~' '#' '@'
#    appear in the path being manipulated.
#    The chk-separator() function will check for the presence of each character in turn
#    until it finds one that does not occur.  If all occur, the rem_* functions will not
#    attempt removal.

# These functions have been written to cope with spaces in the file paths.

# Check each possible regular expression delimiter in turn against the first argument.
# Set the variable dlim to the first delimiter that does NOT occur, or "" if all occur.
chk_delim() {
    local c z
    if [ $# -lt 1 ]; then return; fi
    for c in '|' '%' ',' '~' '#' '@'
    do
        z=`expr "$1" : '.*'"$c"'.*'`
        if [ $z -eq 0 ]; then dlim=$c; break; fi
    done
    if [ "$c" != "$dlim" ]
    then
        dlim=""
    fi
}

# Clear path element given as arg from PATH
# N.B. path element MUST NOT contain a vertical bar `|'
rem_path() {
    if [ $# -lt 1 ]; then return; fi
    rem_vpath PATH "$@"
}

# Clear path element(s) given as arg 2 (3 ...) from the path named in arg1
# N.B. path element MUST NOT contain a vertical bar `|'
rem_vpath() {
    local pathvalue pathname rem_el dlim tmp_path
    if [ $# -lt 2 ]; then return; fi
    eval pathvalue="$$1"
    chk_delim "$pathvalue"
    # If existing pathvalue cannot be edited, return immediately
    if [ "$dlim" == "" ]; then return; fi
    pathname=$1
    shift
    while [ $# -ge 1 ]; do
        rem_el="$1"
        shift
        tmp_path="$pathvalue""$rem_el"
        chk_delim "$tmp_path"
        if [ "$dlim" == "" ]; then continue; fi
        pathvalue=`echo $pathvalue|
        sed ':loop
{s'"$dlim"':'"$rem_el"':'"$dlim"':'"$dlim"'g
t loop
}
s'"$dlim"'^'"$rem_el"':'"$dlim$dlim"'
s'"$dlim"':'"$rem_el"'$'"$dlim$dlim"'
s'"$dlim"'^'"$rem_el"'$'"$dlim$dlim"''`
    done
    eval $pathname="$pathvalue"
}

# To path named in $1, append path element arg(s), replacing all existing
# instances in the named path
# N.B. path elements MUST NOT contain a vertical bar `|'
app_vpath() {
    local pathname pathvalue el
    pathname=$1; shift
    for el
    do
        rem_vpath $pathname "$el"
        eval pathvalue="$$pathname"
        pathvalue="${pathvalue:+${pathvalue}:}$el"
        eval ${pathname}="$pathvalue"
    done
    export $pathname
}

# Append path element(s) given as args to PATH, replacing all existing
# instances in the PATH
# N.B. path elements MUST NOT contain a vertical bar `|'
app_path() {
    app_vpath PATH "$@"
}

# To path named in $1, prepend path element arg(s), replacing all existing
# instances in the named path.  Elements will be appear in the PATH in
# argument order.
# N.B. path elements MUST NOT contain a vertical bar `|'
pre_vpath() {
    local pathname pathvalue sptr element
    pathname=$1; shift
    sptr=0
    while [ $# -gt 0 ]
    do
        eval local elstack$((++sptr))="$1"
        shift
    done
    while [ $sptr -gt 0 ]
    do
        eval element=${elstack$((sptr--))}
        rem_vpath $pathname "$element"
        eval pathvalue="$$pathname"
        pathvalue="$element${pathvalue:+:${pathvalue}}"
        eval ${pathname}="$pathvalue"
    done
    export $pathname
}

# Prepend path element(s) given as args to PATH, replacing all existing
# instances in the PATH.  N.B. elements will be appear in the PATH in
# REVERSE argument order.
# N.B. path elements MUST NOT contain a vertical bar `|'
pre_path() {
    pre_vpath PATH "$@"
}


# Clean a path - run pre_vpath with every current element of the path
# in reverse order
# This removes all duplicates in the path, and leaves the first instance
# of a path element in its original relative place - later ones are deleted.
clean_vpath() {
    local pathname pathvalue
    pathname=$1; shift
    # pathname contains the name of the path
    eval pathvalue="$$pathname"
    # pathvalue contains the value of the path
    pathvalue=`echo "$pathvalue"|sed 's/^/"/
s/:/" "/g
s/$/"/'`
    eval pre_vpath $pathname "$pathvalue"
}

clean_path() {
    clean_vpath PATH
}

# This prints out the elements of the path named in its first argument,
# one per line.
list_vpath() {
    if [ $# -lt 1 ]; then return; fi
    local pathname pathvalue
    pathname=$1; shift
    eval pathvalue="$$pathname"
    echo "$pathvalue"|sed ':loop
{h
s/:.*//
p
g
s/[^:]*://
t loop
d
}'
}

# This prints the elements of PATH, one per line
list_path() {
    list_vpath PATH
}

[ -n "$BASH" ] && 
export -f chk_delim rem_path rem_vpath app_path app_vpath pre_path pre_vpath clean_path clean_vpath list_path list_vpath

Ant: process elements in a list

I was looking for a way to process a list of items in an ant build file, similar to what you would do in Java with a construct like:

for ( Element element : elements ) {
    // do stuff with element
}

The approach of XSLT, using recursive calls with local variables, looked promising. In the ant-user mailing list, I found a posting on the topic Implementing a loop in ANT with something like what I was looking for. Ben Stringer’s example gave me the critical information—that I could make indirecly recursive calls to antcall. He also used an external file, updated with the buildnumber task, to maintain state through the recursive calls. Bingo!

The build file I describe here is available online.

To extract list elements, use the editstring macro described in an earlier post.

Rather than using buildnumber, I use the propertyfile task, which offers a means of passing information down through the recursion, and back to the caller. propertyfile is essentially a property file editor, and is ideal for my purposes.

Setup macros

Setting up the recursion involves

  • creating the list, and assigning it to a property,
  • writing to a property file the properties which will be required in the recursive call.

The setup process is performed by a series of nested macros. The inner macro is string-list, which accepts as attributes a string, the location of a properties file, the target to be antcalled for each list element, an optional flag to keep the properties file after processing, and an optional list separator string. The macro also requires that the element string-list-args be defined by the caller. 

  
<macrodef name="string-list"
    description="List is a string. Default separator is path.separator.">
  <attribute name="properties"
      description="Location of the properties file."/>
  <attribute name="string"
      description="The string to edit."/>
  <attribute name="list.target"
      description="The target to run against each element of the list."/>
  <attribute name="keep-properties" default="false"
      description="Flag to keep the temporary properties file."/>
  <attribute name="separator" default="${path.separator}"
      description="Separator for elements of list."/>
  <element name="string-list-args"/>
  <sequential>
    <property name="list.keep.properties.file.."
        value="@{keep-properties}"/>
    <propertyfile file="@{properties}"
        comment="Automatically generated by string-list.">
      <entry key="list.counter.." type="int" value="0"/>
      <entry key="list" value="@{string}"/>
    </propertyfile>
    <string-list-args/>
    <antcall inheritall="false" target="list-initial__">
      <param name="properties" location="@{properties}"/>
      <param name="list.target" value="@{list.target}"/>
      <param name="separator" value="@{separator}"/>
    </antcall>
    <antcall inheritall="true" target="maybe-del-properties-file__"/>
  </sequential>
</macrodef>

Two other setup macros are provided: union-list and filelist-list. While string-list is general, and accepts any separator string, union-list and filelist-list are specifically for their eponymous types, and assume that ${path.separator} is the separator. filelist-list calls the union-list macro, and union-list calls string-list. Neither will be elaborated on here.

The recursive process

The recursion is started by an antcall to the target list-initial__, with inheritall=”false”. All properties are fed down through the properties file which has already been set up. That is the function of the dependency target list-set-properties__. It reads the properties into the context of this antcall instance. Because inheritall is false, all the properties defined in the file will take on the values in that properties file.

The list is split into the first element and the rest, and the properties file is updated with the first element and the new list.

<target name="list-initial__" depends="list-set-properties__"
  if="list.target" unless="list.finished">
  <!-- Split the list property into first and rest -->
  <editstring string="${list}" toproperty="first"
       pattern="^(.*?)(?&lt;!.*${separator}.*)(${separator}(.+))?$"
       replace="1"/>
  <editstring string="${list}" toproperty="rest"
       pattern="^(.*?)(?&lt;!.*${separator}.*)(${separator}(.+))?$"
       replace="3"/>
  <!-- Set up the properties file for the next (including first) iteration. -->
  <propertyfile file="${properties}">
    <entry key="list.first" value="${first}"/>
    <entry key="list" value="${rest}"/>
  </propertyfile>
  <!-- Start recursive processing -->
  <antcall inheritall="false" target="list-recur__"/>
</target>

The list-initial__ target finally calls list-recur__, again with inheritall=”false”. After reading the properties, and checking for the termination condition, the list element target ${list.target} is antcalled, but with inheritall=”true”, so that the current set of properties is available to it. The informational counter property list.counter.. is incremented in the properties file, and list-initial__ is re-invoked with inheritall=”false”.

<!-- Recursive process. Depends on:
  list-set-properties__
    loads the temporary properties file, containing the properties
    which must vary from invocation to invocation.
  list-recur1__
    checks for termination condition, setting termination flag property.
      
  If list not empty, list-recur__ runs the target against the per-invocation properties,
  then updates the element count in the ${properties} file.

  Finally it recursively calls list-initial__  -->
<target name="list-recur__" depends="list-set-properties__, maybe-list-finished__"
    unless="list.finished">
    <antcall inheritall="true" target="${list.target}"/>
    <propertyfile file="${properties}">
      <entry key="list.counter.." type="int" operation="+" value="1"/>
    </propertyfile>
    <antcall inheritall="false" target="list-initial__"/>
</target>

Usage

This build file, listutils.xml, expects to import editstring.xml from the same directory. It is not necessary to have listutils.xml in the same directory as your project. A build file to test listutils is available here. It also expects to find listutils.xml in the same directory. Here’s part of that file.

<target name="teststringlist">
  <property name="tmp" location="/tmp"/>
  <filelist id="flist1" dir="." files="mbox, errors, frames, frames.xml"/>
  <union id="union1">
    <filelist refid="flist1"/>
  </union>
  <property name="files1" value="${toString:union1}"/>
  <property name="properties" location="${tmp}/recursive.properties"/>
  <string-list properties="${properties}" string="${toString:union1}"
         list.target="list.target" keep-properties="true" separator="pbw">
    <string-list-args>
      <propertyfile file="@{properties}">
        <entry key="argument1" value="Argument 1 value"/>
        <entry key="argument2" value="Argument 2 value"/>
      </propertyfile>
    </string-list-args>
  </string-list>
</target>

<!-- The target to process each element of the list. -->
<target name="list.target">
  <echo>This list element is ${list.first}</echo>
  <echo>Arguments ${argument1} ${argument2}</echo>
</target>

teststringlist creates a filelist, from which it creates a union.  ${toString:<filelist>} does not give the desired result, but ${toString:<union>} returns a list with ${path.separator}s, which is the default separator. In this case, though, I am testing with an alternative separator, which will work for me because the files are all within my home directory, pbw. The other tests in this file assume ${path.separator}. These, of course, will break up the list differently.

Note the element string-list-args within the invocation of string-list. This is defined in the caller, and is expanded within the final expansion of string-list. In this case, and this would be typical, property definitions require in the list.target are added to the properties file, so that they will be available to the target.

The target for each element is here defined as list.target, which happens to be the same as the name of the property that passes it down. That is not necessary, however. In this case, the target simply echoes the list element name, and prints out the properties which have been provided through the string-list-args element.

Here’s the whole listutils.xml file.

<?xml version="1.0"?>
<?xml version="1.0"?>
<project name="listutils">

  <dirname property="listutils.dir" file="${ant.file.listutils}"/>
  <import file="${listutils.dir}/editstring.xml"/>
  
  <macrodef name="string-list"
      description="List is a string. Default separator is path.separator.">
    <attribute name="properties" description="Location of the properties file."/>
    <attribute name="string" description="The string to edit."/>
    <attribute name="list.target"
        description="The target to run against each element of the list."/>
    <attribute name="keep-properties" default="false"
        description="Flag to keep the temporary properties file."/>
    <attribute name="separator" default="${path.separator}"
        description="Separator for elements of list."/>
    <element name="string-list-args"/>
    <sequential>
      <property name="list.keep.properties.file.." value="@{keep-properties}"/>
      <propertyfile file="@{properties}" comment="Automatically generated by string-list.">
        <entry key="list.counter.." type="int" value="0"/>
        <entry key="list" value="@{string}"/>
      </propertyfile>
      <string-list-args/>
      <antcall inheritall="false" target="list-initial__">
        <param name="properties" location="@{properties}"/>
        <param name="list.target" value="@{list.target}"/>
        <param name="separator" value="@{separator}"/>
      </antcall>
      <antcall inheritall="true" target="maybe-del-properties-file__"/>
    </sequential>
  </macrodef>
  
  <macrodef name="union-list"
            description="List is result of ${toString:[union-id]}. Separator is path.separator.">
    <attribute name="properties" description="Location of the properties file."/>
    <attribute name="union-id" description="Id of a union containing the list."/>
    <attribute name="list.target"
               description="The target to run against each element of the list."/>
    <attribute name="keep-properties" default="false"
               description="Flag to keep the temporary properties file."/>
    <element name="union-list-args"/>
    <sequential>
      <string-list properties="@{properties}" string="${toString:@{union-id}}"
                   list.target="@{list.target}" keep-properties="@{keep-properties}">
        <string-list-args>
          <union-list-args/>
        </string-list-args>
      </string-list>
    </sequential>
  </macrodef>
  
  <macrodef name="filelist-list"
            description="List is a filelist. Must be converted to a union.">
    <attribute name="properties" description="Location of the properties file."/>
    <attribute name="filelist-id" description="Id of a filelist containing the list."/>
    <attribute name="list.target"
               description="The target to run against each element of the list."/>
    <attribute name="keep-properties" default="false"
               description="Flag to keep the temporary properties file."/>
    <element name="filelist-list-args"/>
    <sequential>
    <!-- The list is constructed by taking a <union> of a <filelist>.
        Union is required, because the ${toString:<propertyname>} expansion does
        not give the contents when applied to a <filelist>. When used on a <union>,
        it returns a text list separated by ${path.separator}.
    -->
      <union id="flist-union">
        <filelist refid="@{filelist-id}"/>
      </union>
      <union-list properties="@{properties}" union-id="flist-union"
                  list.target="@{list.target}" keep-properties="@{keep-properties}">
        <union-list-args>
          <filelist-list-args/>
        </union-list-args>
      </union-list>
    </sequential>
  </macrodef>

  <target name="list-element-count__">
    <property file="${properties}"/>
    <echo>Number of elements processed: ${list.counter..}</echo>
  </target>

  <!-- Clean up the temporary properties file. -->
  <target name="maybe-del-properties-file__" depends="list-element-count__"
          unless="${list.keep.properties.file..}">
    <delete file="${properties}"/>
  </target>

  <target name="list-initial__" depends="list-set-properties__" if="list.target" unless="list.finished">
    <!-- Split the list property into first and rest -->
    <editstring string="${list}" toproperty="first"
                   pattern="^(.*?)(?&lt;!.*${separator}.*)(${separator}(.+))?$"
                   replace="1"/>
    <editstring string="${list}" toproperty="rest"
                   pattern="^(.*?)(?&lt;!.*${separator}.*)(${separator}(.+))?$"
                   replace="3"/>
    <!-- Set up the properties file for the next (including first) iteration. -->
    <propertyfile file="${properties}">
      <entry key="list.first" value="${first}"/>
      <entry key="list" value="${rest}"/>
    </propertyfile>
    <!-- Start recursive processing -->
    <antcall inheritall="false" target="list-recur__"/>
  </target>

  <!-- Recursive process. Depends on:
      list-set-properties__ loads the temporary properties file, containing the properties
                            which must vary from invocation to invocation.
      list-recur1__ checks for termination condition, setting termination flag property.
      
      If list not empty, list-recur__ runs the target against the per-invocation properties,
      then updates the element count in the ${properties} file.

      Finally it recursively calls list-initial__  -->
  <target name="list-recur__" depends="list-set-properties__, maybe-list-finished__"
          unless="list.finished">
    <antcall inheritall="true" target="${list.target}"/>
    <propertyfile file="${properties}">
      <entry key="list.counter.." type="int" operation="+" value="1"/>
    </propertyfile>
    <antcall inheritall="false" target="list-initial__"/>
  </target>

  <!-- Load the property file to obtain the properties for this invocation -->
  <target name="list-set-properties__">
    <property file="${properties}"/>
  </target>

  <!-- Set the property list.finished if the list is exhausted. -->
  <target name="maybe-list-finished__">
    <condition property="list.finished">
      <equals arg1="${list.first}" arg2="$${rest}"/>
    </condition>
  </target>

</project>

Ant: edit property values

One of the frustrations of using ant was the difficulty of deriving one property value performing some sort of editing operation on an existing property value. The mapper task does a lot of grunt work for file names, but not for property values as such.<.p>

A common requirement is to map a Java package name to a corresponding directory structure. I have a property containing the package name, and I want to create another property with the directories. Here’s one way to do that.

<property name="my.package" value="my.java.package"/>
<!-- ... Some time later ... -->
<loadresource property="my.package.dirs">
<string value="${my.package}"/>
<filterchain>
<replacestring from="." to="${file.separator}"/>
<striplinebreaks/>
      </filterchain>
</loadresource>

For more complex transformations, a replaceregexp filter can be used. The above example would then be:

<property name="my.package" value="my.java.package"/>
<!-- ... Some time later ... -->
<loadresource property="my.package.dirs">
<string value="${my.package}"/>
<filterchain>
<striplinebreaks/>
<replaceregex pattern="." replace="${file.separator}" flags="g"/>
</filterchain>
</loadresource>

Here’s a macrodef to perform string editing, included in an ant build file called editstring.xml with a test invocation. The base file, without the test, can be downloaded here.

<project name="editstring">
  <macrodef name="editstring"
    description="Edit a string using a regexp pattern and repacement, with optional flags, placing result in a property.">
    <attribute name="string"
      description="String being edited."/>
    <attribute name="dest"
      description="Name of the property receiving the edit result."/>
    <attribute name="pattern"
      description="Regexp pattern to be replaced."/>
    <attribute name="replace"
      description="Regexp replacement pattern."/>
    <attribute name="flags"
      description="Regexp replacement flags."
      default="-g"/>
    <sequential>
      <loadresource property="@{dest}">
        <string value="@{string}"/>
          <filterchain>
            <replaceregex pattern="@{pattern}"
             replace="@{replace}" flags="@{flags}"/>
            <striplinebreaks/>
          </filterchain>
      </loadresource>
    </sequential>
  </macrodef>

  <target name="junk">
    <property name="teststr" value="My test string."/>
    <editstring string="${teststr}" dest="result"
      pattern="^(S+s+)S+(s+.*)$" replace="1result2"/>
    <echo>${teststr} : ${result}</echo>
  </target>
</project>

Be aware of the striplinebreaks filter. I found when I used this method that I was getting a spurious newline on the end of my edited string, so I inserted striplinebreaks. However, if you are editing a multiline string, this will break your replacement. You will have to experiment in those circumstances. I suspect that this is an artifact of the line-at-a-time processing of replacement text. I think it appends a line break after replacement.

I first read about this solution in a post by Mark Melvin. Thanks to Mark.

Using environment.plist with Mountain Lion

UPDATE
This post is now obsolete. For the preferred method in both Mountain Lion and Mavericks, see  Setting environment variables in OS X Mountain Lion and Mavericks. With Mountain Lion (OS X 10.8), the environment settings from ~/.MacOSX/environment.plist are not taken into account when the background system environment is set up by launchd, the OS X initialisation process. Consequently, if you want settings like $JAVA_HOME to be available to Java applications you start from Finder, you must find an alternative for setting them.

The beauty of the environment.plist method was that, by setting the variables in the plist from within your .profile, you could keep the shell and system environments in sync. There are obvious advantages in this synchronisation. If you are like me, you have gone to a lot of trouble to put such synchronisation in place. See my previous post, Setting Environment Variables in OS X Lion.

The now-recommended method of system variable setting is to use the setenv subcommand of launchctl, like so:

launchctl setenv M2 /usr/share/maven/bin

Such commands can be saved in the file /etc/launchd.conf, in which case only the subcommand is saved:

setenv M2 /usr/share/maven/bin

You can check the state of the launchd environment with the subcommand export:

launchctl export

This will output a series of shell commands of the form

M2=" /usr/share/maven/bin"; export M2;

This serves as a test for any attempts to set the launchd environment variables.

There are three problems with using launchctl in this way. Firstly, /etc/launchd.conf is a system file (when it exists, which it does not by default), and it is in a directory (/etc) which is writable only by root. Yes, you can sudo to edit the file, but then you face the second problem, touched on above. You have two discrete sources for environment variable settings, which can easily get out of sync. Lastly, the strings defined in setenv statements like the above, must be string literals. There is no means of resolving variable content based on other variables. For example, you cannot define

setenv TMPDIR $HOME/tmp

The problem, then, is to find a way to dynamically set the launchd environment, based on your existing .profile and .MacOSX/environment.plist settings. Another bonus from this approach is that, within you .profile, you can use all of the shell facilities to define the content of variables.

When the system starts, the launchd process finds system daemon processes to launch from /System/Library/LaunchDaemons & /Library/LaunchDaemons. When a user logs in, launchd looks for user agents to start up in /System/Library/LaunchAgents, /Library/LaunchAgents & ~user/Library/LaunchAgents. While the first two are system-owned directories, the third is owned by the individual user.
We need two files: 1) a shell file which will actually issue the launchctl setenv commands, and 2) a plist file that will tell launchd where to find the executable, and how to run it. The shell file is available here; the plist file is available here.

The executable: plist2launchctl

#!/bin/sh

PLIST="$HOME/.MacOSX/environment.plist"
SLEEP_TIME=10
# Uncomment following to echo launchctl commands to stdout
# Key StandardOutPath will have to be set in the plist file
# (au.id.pbw.plist2launchctl) for output to be captured.
#ECHO_TO_STDOUT=true

one_cmd () {
    eval "$@"
}

[[ -e "$SHFILE" ]] && {
    rm "$SHFILE"
    [[ $? -eq 1 ]] && {
        echo "$SHFILE" cannot be removed. >&2
        exit 1
    }
}

cmd_list=`cat "$PLIST" | 
sed -En 
'/^[[:space:]]*<key>(.*)</key>[[:space:]]*/ {
s//launchctl setenv 1 /
N
}
/^.*<string>.*[[:space:]].*</string>.*$/ {
d
}
/^(launchctl .*)n[[:space:]]*<string>(.+)</string>[[:space:]]*$/ {
s//12/
p
d
}'`

[[ -n "$ECHO_TO_STDOUT" ]] && echo "$cmd_list"

echo "$cmd_list" | while read line; do one_cmd $line; done

[[ -n "$ECHO_TO_STDOUT" ]] && echo Sleeping in plist2launchctl

# Sleep for a while so launchd doesn't get upset
[[ -n "$SLEEP_TIME" ]] && sleep "$SLEEP_TIME"

Notes:
The output of the whole of the cat|sed pipeline is captured in the cmd_list variable by enclosing the lot in backquotes. The sed command is itself enclosed in single quotes. It simply reads the contents of environment.plist line at a time, and merges key/string pairs into the corresponding launchctl setenv commands, storing them in cmd_list. cmd_list is then itself read line at a time, and each line is handed to eval for execution.
Permissions & Location:
To stay on the safe side, give the file -rwxr-xr-x permissions. It should be placed in /usr/libexec, which is root owned. You will have to use sudo to copy it.

The plist file: au.id.pbw.plist2launchctl

<? xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
    <dict>
        <key>Label</key>
        <string>au.id.pbw.plist2launchctl</string>
        <key>KeepAlive</key>
        <false/>
        <key>Program</key>
        <string>/usr/libexec/plist2launchctl</string>
        <key>RunAtLoad</key>
        <true/>
        <key>UserName<key>
        <string>pbw</string>
    </dict>
</plist>
Notes:
The UserName field will, of course, be set to your own login name.
If you need to debug the plist file, add the following lines.

        <key>StandardOutPath</key>
        <string>/Users/pbw/plist.log</string>
        <key>Debug</key>
        <true/>
These lines are used in conjunction with the ECHO_TO_STDOUT variable in the executable file. If used, the StandardOutPath will point to a writable file on your system.
Permissions & Location:
Again, give the file -rwxr-xr-x permissions. Place it in your $HOME/Library/LaunchAgents directory.