refusal computing

Reese Erlich is a freelance journalist and author who's been covering recent events in Iran -- *from Iran* and not just his computer chair like many in the mainstream media have.

He recently countered "left-wing Doubting Thomas arguments" in an article on Common Dreams .org. In his arguments, I found these observations about Iranians "fighting for political, social and economic justice" inspiring.

[...]

Assertion: The U.S. has a long history of meddling in Iran, so it must be behind the current unrest.

[...]

Frankly, based on my observations, no one was leading the demonstrations. During the course of the week after the elections, the mass movement evolved from one protesting vote fraud into one calling for much broader freedoms. You could see it in the changing composition of the marches. There were not only upper middle class kids in tight jeans and designer sun glasses. There were growing numbers of workers and women in very conservative chadors.

Iranian youth particularly resented President Ahmadinejad's support for religious militia attacks on unmarried young men and women walking together and against women not covering enough hair with their hijab. Workers resented the 24 percent annual inflation that robbed them of real wage increases. Independent trade unionists were fighting for decent wages and for the right to organize.

Some demonstrators wanted a more moderate Islamic government. Others advocated a separation of mosque and state, and a return to parliamentary democracy they had before the 1953 coup. But virtually everyone believes that Iran has the right to develop nuclear power, including enriching uranium. Iranians support the Palestinians in their fight against Israeli occupation, and they want to see the U.S. get out of Iraq.

So if the [sic] CIA was manipulating the demonstrators, it was doing a piss poor job.

[...]

See also Erlich's Iran is not a twitter revolution.

I've written before how I believe that a lot of women aren't recruited or drawn into all levels of computing because the culture is male-centered and therefore not attractive to women. I also believe a symptom of this disease -- that also only adds to make the situation even worse -- are the outright sexist and misogynistic acts by men. Often, such harassment is acted out in private or in electronic forums -- these incidents are well-documented. However, occasionally this hostility bubbles up and boils over into public and in-person situations.

This month, there was a presentation at a Flash conference that sounded completely absurd. Read about it at Prude or Professional? by Courtney Remes at the geekgirlsguide.

Earlier this year there was a similar presentation at a Ruby conference. See Why Rails is Still Ghetto by sarahmei and gender and sex at gogaruco by Sarah Allen for reports from a third of the women who attended the conference (2 out of 6 is one-third) and found the presentation offensive.

On all this, I prefer to just quote some of what volsunga wrote in Porn. Ruby. *headdesk* rather than adding more meta-discussion to these controversies. It is stated well.

[...] This is the classic "it's more offensive for you to say I'm a sexist than for me to actually be sexist!" response. People with an agenda (usually those sneaky feminists) choose to find something offensive so they can have a whine and call someone mean names, like "sexist". But what's at stake here isn't that the presentation was offensive per se, but that the context was inappropriate and potentially alienating to women developers, in an environment that's already default male by dint of numbers.

There's also the classic "you could just ignore it if you don't like it" defence. [...]

This presumes that people who don't like pictures of naked women went along just so they could complain. But even if everyone who thought they might not like the talk didn't go, it'll still be wrong to show it; the very presence of such a slideshow at the event creates an atmosphere where women are "them", where some content is made solely for men, but as if "male" is "default". [...]

And it doesn't matter if it was intentional -- no one really thinks [the presenter] sat down and schemed to offend women in advance -- and by refocusing on intention [the presenter] is able to get away with all that "poor little me" stuff in his post, as if his whole character has been impugned.

Newsflash: there's a difference between saying "you're a sexist/racist/homophobe" and "some of the stuff you just did/said contributed to the sexist/racist/homophobic culture around X".

Message to Ruby developers who think this is out of control/proportion/just a bit silly: all your rights to nod sympathetically/join in when someone bemoans the lack of women developers are entirely removed (for ever) if when women do speak up, you pull this self-pitying, I'm-a-nice-guy-really, its-not-my-fault, thats-just-the-way-I-roll, stop-complaining bullshit. And if those who complained then get painted as moralistic, shrill and angry for the sake of it.

There are various posts up and around about why this has become a blame game, and that it's counter-productive. It wouldn't be a blame game if there had been less bombastic denial and more listening on the part of the speaker in the first place. Blame games stop when someone puts their hands up and scrutinises their behaviour. So get on with it.

A fun resource I found while poking around in this is Derailing for Dummies.

I noticed an old comment posted to someone's site by Matthew Davidson who wrote a car metaphor for free software. This is a commonly used metaphor device, but I think it's especially good. (I *swear* I didn't find this in a search for my own surname).

I drive a little Toyota hatchback. I do so because I got it relatively cheap from my sister-in-law. This was possible because she owned the car, [instead of having] a non-transferable license to use the car under certain conditions.

I know how to pump up the tyres and refill the thing that squirts water on the windscreen. That's all I know about maintaining the vehicle, and probably all I ever will know. I take it to the local mechanic of my choice every couple of years and he fustigates the Smoot-Hawley flanges or whatever for me, at what I can only assume is a reasonable price.

I am very glad that the bonnet was not locked shut at the factory by Toyota, and that there is not a Drivable Motor Vehicle Act (DMVA) to make it a criminal offense for anybody to attempt to service their own car, or pay somebody other than the manufacturer to service it. I may not personally know the first thing about its [sic] inner workings, but if I suspect I'm being charged to much for some work on my car, I can go a few hundred metres up the road to the next mechanic who can provide me with a quote.

Most of these mechanics probably chose this trade after opening up the bonnet of their own car and having a playful poke around, the same way I learned how to program computers. Now as Richard Stallman would say, the ethical issues around car manufacturing and software manufacturing are not the same; I don't have the legal right to make a perfect copy of my car, but that's okay because I don't have the practical means to do so -- no matter how much technical skill I am able to acquire, and neither does anybody but very large corporations, so losing that freedom (through patents) doesn't cost me anything, while potentially delivering the benefits to society that the patent system [for car manufacturing] is supposed to provide.

But if somebody paid me to write some software for them and I said "okay, I'll write it for you, but only under the condition that you don't copy it or attempt to fix or improve it yourself, or pay somebody else to fix or improve it," that would be a very bad deal for the customer, because the means to do these things are so cheap that you are practically only paying for the time of the person who does the work (or not, if you do it yourself). It would be such a bad deal in fact, that if I managed to convince a sucker to fall for it, I would have to regard my own behaviour as unethical.

Granted there aren't as many programmers as motor vehicle mechanics in my town, but that can and should change. Already I can point to half a dozen people I know who could (and hopefully will) become as familiar with the inner workings of [the free software package] Drupal as myself with only a little effort. As this begins to happen across a wide range of software the real cost of proprietary software (as opposed to the mere price tag), and the benefits of freedom, will become apparent to even the most non-technical users.

Speaking of "tryes", "metres" and "bonnets"; the end software patents campaign needs help documenting the patent issue in Australia and New Zealand, among other locales.

The dump/restore package version 0.4b42 is being released today. It is the first release in over 3 years. Read the release notes.

It supports versions two through four of the Linux kernel's extended file system (ext2, ext3, ext4). Technically, it is considered beta software, but I have reason to believe there are a lot of people who use it in production systems. You can also use it for disk-based backups. You don't need to have a tape jukebox to use it, although it is designed for use with tape backups.

With Unicode characters, in this case the popular UTF-8, sometimes you need to convert characters to ASCII to get things done in PHP. In the case of sorting Unicode, there are the existing solutions of collator_sort() for PHP5 and strcoll() since PHP4. However, they both assume a locale. A hack that is locale-agnostic would just "normalize" Unicode characters to ASCII.

This is far from complete, but seems to do the right thing.

    <?php

    /**
     * Normalize international characters for purposes like sorting and
     * searching by using a heuristic that just uses ASCII--the english
     * alphabet ordering--for a multilingual solution--no locale setting.
     */
    header("Content-type: text/plain; charset=utf-8");

    /**
     * Iñtërnâtiônàlizætiøn
     *
     * Example from Sam Ruby
     * http://intertwingly.net/stories/2004/04/14/i18n.html
     * 
     * By way of WACT team
     * http://www.phpwact.org/php/i18n/charsets
     */
    $internationalization = array(
				  "I", // I
                                  "\xC3\xB1", // ñ
                                  "t", // t
                                  "\xC3\xAB", // ë
                                  "r", // r
                                  "n", // n
                                  "\xC3\xA2", // â
                                  "t", // t
                                  "i", // i
                                  "\xC3\xB4", // ô
                                  "n", // n
                                  "\xC3\xA0", // à
                                  "l", // l
                                  "i", // i
                                  "z", // z
                                  "\xC3\xA6", // æ
                                  "t", // t
                                  "i", // i
                                  "\xC3\xB8", // ø
                                  "n"); // n
    
    /** 
     * Use strtr() with this dictionary to convert to ASCII.
     * This data structure is not comprehensive.
     */
    $utf8_dict = array("\xC3\x80" => "A", // À
                       "\xC3\x81" => "A", // Á
                       "\xC3\x82" => "A", // Â
                       "\xC3\x83" => "A", // Ã
                       "\xC3\x84" => "A", // Ä
                       "\xC3\x85" => "A", // Å
                       "\xC3\x86" => "A", // Æ
                       "\xC3\x9E" => "B", // Þ
                       "\xC3\x87" => "C", // Ç
                       "\xC4\x86" => "C", // Ć
                       "\xC4\x8C" => "C", // Č
                       "\xC4\x90" => "Dj", // Đ
                       "\xC3\x88" => "E", // È
                       "\xC3\x89" => "E", // É
                       "\xC3\x8A" => "E", // Ê
                       "\xC3\x8B" => "E", // Ë
                       "\xC4\x9E" => "G", // Ğ
                       "\xC3\x8C" => "I", // Ì
                       "\xC3\x8D" => "I", // Í
                       "\xC3\x8E" => "I", // Î
                       "\xC3\x8F" => "I", // Ï
                       "\xC4\xB0" => "I", // İ
                       "\xC3\x91" => "N", // Ñ
                       "\xC3\x92" => "O", // Ò
                       "\xC3\x93" => "O", // Ó
                       "\xC3\x94" => "O", // Ô
                       "\xC3\x95" => "O", // Õ
                       "\xC3\x96" => "O", // Ö
                       "\xC3\x98" => "O", // Ø
                       "\xC3\x9F" => "Ss", // ß
                       "\xC3\x99" => "U", // Ù
                       "\xC3\x9A" => "U", // Ú
                       "\xC3\x9B" => "U", // Û
                       "\xC3\x9C" => "U", // Ü
                       "\xC3\x9D" => "Y", // Ý
                       "\xC3\xA0" => "a", // à
                       "\xC3\xA1" => "a", // á
                       "\xC3\xA2" => "a", // â
                       "\xC3\xA3" => "a", // ã
                       "\xC3\xA4" => "a", // ä
                       "\xC3\xA5" => "a", // å
                       "\xC3\xA6" => "a", // æ
                       "\xC3\xBE" => "b", // þ
                       "\xC3\xA7" => "c", // ç
                       "\xC4\x87" => "c", // ć
                       "\xC4\x8D" => "c", // č
                       "\xC4\x91" => "dj", // đ
                       "\xC3\xA8" => "e", // è
                       "\xC3\xA9" => "e", // é
                       "\xC3\xAA" => "e", // ê
                       "\xC3\xAB" => "e", // ë
                       "\xC3\xAC" => "i", // ì
                       "\xC3\xAD" => "i", // í
                       "\xC3\xAE" => "i", // î
                       "\xC3\xAF" => "i", // ï
                       "\xC3\xB0" => "o", // ð
                       "\xC3\xB1" => "n", // ñ
                       "\xC3\xB2" => "o", // ò
                       "\xC3\xB3" => "o", // ó
                       "\xC3\xB4" => "o", // ô
                       "\xC3\xB5" => "o", // õ
                       "\xC3\xB6" => "o", // ö
                       "\xC3\xB8" => "o", // ø
                       "\xC5\x94" => "R", // Ŕ
                       "\xC5\x95" => "r", // ŕ
                       "\xC5\xA0" => "S", // Š
                       "\xC5\x9E" => "S", // Ş
                       "\xC5\xA1" => "s", // š
                       "\xC3\xB9" => "u", // ù
                       "\xC3\xBA" => "u", // ú
                       "\xC3\xBB" => "u", // û
                       "\xC3\xBC" => "u", // ü
                       "\xC3\xBD" => "y", // ý
                       "\xC3\xBD" => "y", // ý
                       "\xC3\xBF" => "y", // ÿ
                       "\xC5\xBD" => "Z", // Ž
                       "\xC5\xBE" => "z"); // ž
    
    $i18n = join("", $internationalization);
    print $i18n . "\n";

    /**
     * UTF-8 regular expression from
     * http://php.net/manual/en/function.utf8-decode.php (comment 57069)
     */
    $utf8_re = "/^([\\x00-\\x7f]|"
      . "[\\xc2-\\xdf][\\x80-\\xbf]|"
      . "\\xe0[\\xa0-\\xbf][\\x80-\\xbf]|"
      . "[\\xe1-\\xec][\\x80-\\xbf]{2}|"
      . "\\xed[\\x80-\\x9f][\\x80-\\xbf]|"
      . "\\xef[\\x80-\\xbf][\\x80-\\xbc]|"
      . "\\xee[\\x80-\\xbf]{2}|"
      . "\\xf0[\\x90-\\xbf][\\x80-\\xbf]{2}|"
      . "[\\xf1-\\xf3][\\x80-\\xbf]{3}|"
      . "\\xf4[\\x80-\\x8f][\\x80-\\xbf]{2})*$/";

    print "Valid UTF-8?: " . (preg_match($utf8_re, $i18n) > 0
			      ? "true" : "false") . "\n";

    print strtr($i18n, $utf8_dict) . "\n";

    // Doesn't work in PHP4?
    $sorted = preg_split("//u", $i18n, -1, PREG_SPLIT_NO_EMPTY);
    // So, just use the original array, instead.
    $sorted = $internationalization;

    function compare($s1, $s2)
    {
      global $utf8_dict;
      return strcasecmp(strtr($s1, $utf8_dict),
			strtr($s2, $utf8_dict));
    }

    usort($sorted, "compare");
    print join("", $sorted) . "\n";

    /**
     * Results:
     * 
     * Iñtërnâtiônàlizætiøn
     * Valid UTF-8?: true
     * Internationalization
     * àæâëIiiilñnnnøôrtttz
     */
    ?>

I tried the I18N_UnicodeNormalizer from the PHP PEAR project, and it didn't do what I wanted.

    <?php

    require_once('I18N/UnicodeNormalizer.php');

    print I18N_UnicodeNormalizer::toNFD($i18n) . "\n";
    print I18N_UnicodeNormalizer::toNFC($i18n) . "\n";
    ?>

There's a good chance I don't know what I'm doing there with the PEAR library, however.

In Emacs, insert UTF-8 hex value for a PHP string of the character at point.

(defun php-hex-for-char ()
  (interactive)
  (insert
   (mapconcat (lambda (x) (format "\\x%02X" x))
              (encode-coding-char (char-after (point)) 'utf-8)
              "")))

Lisp lifted from `describe-char' and `encoded-string-description'.

I came across a project whose source code contains both DOS text files and Unix text files. Some of the Unix files contain carriage return line endings. Though, perhaps they were DOS files with Unix end lines! I wanted to suggest converting those files with mixed line endings to Unix.

Sometimes, the file command is helpful for showing what files have a mixed end of line style, but not always. For example, the file command will say "ASCII C program text, with CRLF, LF line terminators". That's perfect. However, sometimes the command just says, "PHP script text".

I wrote this find expression that would get files that contain DOS carriage returns, but not entirely DOS files.

$ find -type f -execdir grep -qe '^V^M$' {} \; \
       ! -execdir awk 'BEGIN{is_dos=1;}!/\r$/{is_dos=0}END{exit(!is_dos);}' {} \; \
       -print

The above doesn't work, since many DOS files don't end in a newline (and without a carriage return) as they do for Unix text files.

Awk obviously considers the last line as a line, but since there's no carriage return the file is not considered a DOS file based on the logic I've written. This results in a false negative.

This change to the Awk script makes this hack work as it should.

$ find -type f -execdir grep -qe '^V^M$' {} \; \
       ! -execdir awk 'BEGIN{is_dos=1;}
                       !/\r$/ && is_dos{is_dos=0;n=NR}
                       END{exit(!is_dos && n != NR);}' {} \; \
       -print

I just noticed there was a great action by the FSF's anti-DRM campaign last month against Amazon's Kindle electronic book. See Defective by Design: Impeach Bezos for Amazon's Kindle Swindle. The idea is to send baby food peaches to Jeff Bezos for having terms that allow Amazon to deny customer's access to read their electronic books.

Sending baby food is pretty easy in the age of the Internet, see the link and instructions at the end of the post.

Someone asked me if there was a good way to annotate the changes of an HTML file. It sounded like the person had to maintain some legacy, HTML-hell, home-brewed, template files for some business Web site.

I suggested using the ChangeLog support of Emacs, and using HTML comments to organize sections of an HTML source file. Here's a simple, made-up example of such an HTML file.

<html>
<head>
<title>Sample only</title>
</head>
<body>
<!-- begin header -->
<p>[ <a id="top" href="#bottom">bottom</a> ]</p>
<!-- end header -->
<h1>Sample title</h1>
<!-- BEGIN: PAGE_CONTENT -->
<div>
<p>Testing.</p>
</div>
<!-- END: PAGE_CONTENT --
  -- footer-bottom start -->
<p>[ <a id="bottom" href="#top">top</a> ]</p>
<!-- footer-bottom end -->
</body>
</html>

Unfortunately, support for either the above sectioning style, or even another alternative, is not provided by the HTML mode that ships with Emacs. This is understandable because there is no consistent standard of doing this, and people use other variations than even those covered in the example. Not to mention, HTML comments are used for other reasons than naming regions of the file.

Regardless, I've put together the following regular expression for add-log-current-defun-header-regexp. It handles the cases in the example above. It is set for all buffers using HTML mode. Just put the following in your .emacs file.

    (add-hook 'html-mode-hook
        (lambda ()
          (make-local-variable
           'add-log-current-defun-header-regexp)
           (setq add-log-current-defun-header-regexp
               (concat "^[ \t]*<?!?--[ \t]*\\(?:begin\\|BEGIN\\|start\\)?"
                       "[ \t:]*\\([-_[:alnum:]]+\\)"
                       "[ \t]*\\(?:begin\\|BEGIN\\|start\\)?[ \t]*--"))))

Use it by typing `C-x 4 a' (add-change-log-entry-other-window). An entry like the following will be added in a nearby ChangeLog file:

2009-05-05  Aaron S. Hawley  <aaronhawley@livejournal.com>

        * file.html (PAGE_CONTENT): Add a test paragraph.
        (footer-bottom): Added link to "#top".

This setup will work for most cases except for scenarios where there is nested sectioning or where you've run `C-x 4 a' from a point outside of a "section" and get a false-positive.

To avoid using a builtin command of a Bourne or Bash shell in a shell script, one can use the full path of the executable command. For example, rather than

$ echo Hello, World\!
Hello, World!

you could

$ /bin/echo Hello, World\!
Hello, World!

Here's a way to show the difference--and make fun of the GNU coding standards at the same time.

$ echo --version
--version

$ /bin/echo --version
echo (GNU coreutils) 6.12
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3 : GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Brian Fox and Chet Ramey.

I prefer to use exec than using the full path for a command so that the PATH environment variable is used, and avoid the day should the full path to a binary change some day.

Unfortunately, a consequence of exec is that it runs the command in the current process and therefore will exit on completion, thus cutting short the life of your shell script. To avoid that, just wrap an exec statement in a sub-shell by using parens:

$ ( exec echo --version )
echo (GNU coreutils) 6.12
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3 : GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Brian Fox and Chet Ramey.

I have never seen this written in a script before. Perhaps, there's another way--that's a bit more canonical--to do this. This construct is entirely redundant and contradictory--"exec something in the current shell, but also in a sub-shell". Further, it's probably pretty much always the case to opt for the shell built-in. There are zero to no cases where you want to avoid the built-in. My only scenarios are timing processes in the shell.

According to the Limitations of Shell Builtins section of the GNU Autoconf manual,

When it is desired to avoid a regular shell built-in, the workaround is to use some other forwarding command, such as env or nice, that will ensure a path search:

          $ pdksh -c 'exec true --version' | head -n1

          $ pdksh -c 'nice true --version' | head -n1
          true (GNU coreutils) 6.10
          $ pdksh -c 'env true --version' | head -n1
          true (GNU coreutils) 6.10
     

That manual has everything it it. I guess I'll go with env, doesn't sound as nice as "exec", but it's a good mnemonic since it use the environment's path variable to run the command.

$ env echo --version
echo (GNU coreutils) 6.12
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Brian Fox and Chet Ramey.

Needed to make some Apache redirects for some links on a unix user's group Web site I maintain. The new site is based in a Wiki, and a member of the group moved all the pages with meeting announcements by hand using more readable page names. The old pages had the data as a four-digit year, two-digit month followed by the two-digit day (for example, 20061219). The new pages have the spelled out version of the week day and month (for example, Tuesday, December 19, 2006).

Here's a sample of what I needed for the .htaccess file.

Redirect /group/meeting-20061219.html   http://host.org/group/wiki/index.php/Tuesday,_December_19,_2006
Redirect /group/meeting-20070417.html   http://host.org/group/wiki/index.php/Tuesday,_April_17,_2007
Redirect /group/meeting-20070515.html   http://host.org/group/wiki/index.php/Tuesday,_May_15,_2007
Redirect /group/meeting-20070717.html   http://host.org/group/wiki/index.php/Tuesday,_July_17,_2007
Redirect /group/meeting-20071128.html   http://host.org/group/wiki/index.php/Wednesday,_November_28,_2007
Redirect /group/meeting-20080618.html   http://host.org/group/wiki/index.php/Wednesday,_June_18,_2008

I could do this by-hand, but I'd rather get a shell script to do it right, the first time. I found it easy to do with an extended Grep expression, awk and the date command that comes with GNU coreutils.

$ ls -1 \
  | grep -Ee '[0-9]{8}.html$' \
  | perl -pe 's/([0-9]{4})([0-9]{2})([0-9]{2}).html$/$&\t\1-\2-\3/' \
  | awk '{printf $1 "\t";
          system("date +\"%A,_%B_%e,_%Y\" -d "  $2);}' \
  | awk '{print "Redirect", "/group/" $1,
                "http://host.org/group/wiki/index.php/" $2;}'

I'm thankful I consistently used a file naming convention with the old site.

This simple bit of PHP made some changes to a MediaWiki installation for me recently. Don't use it. My point here is in showing the satisfaction from changing a database with code that generates the SQL statements for you. The golden rule for database applications is to make sure there's an interface or programming layer to the actual database to preserve data integrity and keep changes limited in scope. One should use some of the scripts that come with MediaWiki--batchMove.php or namespaceDupes.php for example--to do this. Such scripts are better trusted, but also handle the schema should it change in a later release of the software.

Now that the disclaimer is out of the way, I was pleasantly surprised how consistent the schema for MediaWiki was for allowing me to rely on simple data structures and some for-loops to update 9 tables. Clearly, the schema isn't entirely normalized, but I predict there is a rationale for having some of the data denormalized. Given this scenario, it is a real compliment to a software package and its schema if one can write very concise code to generate a series of SQL statements for a task.

<?php
// Move 3 pages, and start 2 namespaces.  Don't use this code!

$table = array('page' => 'mw_page',
	       'rc' => 'mw_recentchanges',
	       'pl' => 'mw_pagelinks',
	       'pt' => 'mw_protected_titles',
	       'qc' => 'mw_querycache',
	       'qcc' => 'mw_querycachetwo',
	       'rd' => 'mw_redirect',
	       'tl' => 'mw_templatelinks',
	       'wl' => 'mw_watchlist');

$namespaces = array(100 => 'ThisWiki',
		    /* 101 => 'ThisWiki talk', */
		    110 => 'RPM',
		    /* 111 => 'RPM talk' */);

$rename = array('Changes_to_Wiki_database' => 'ThisWiki:Changes_to_database',
		'LocalSettings.php' => 'ThisWiki:LocalSettings.php',
		'Changes_to_Monobook_skin'
		  => 'ThisWiki:Changes_to_Monobook_skin');

foreach ($table as $short => $t) {
  // Move:
  foreach ($rename as $orig => $new) {
    foreach (array(0 => '', 1 => 'Talk') as $old_ns => $old_name) {
      printf("UPDATE %s\n"
	     . "SET %s_title = '%s'\n"
	     . "WHERE %s_title = '%s' AND %s_namespace = %d;\n",
	     $t, $short, $new, $short, $orig, $short, $old_ns);
    }
  }
  // Fix namespace:
  foreach ($namespaces as $num => $name) {
    foreach (array(0 => '', 1 => 'Talk') as $old_ns => $old_name) {
      printf("UPDATE %s\n"
	     . "SET %s_title = REPLACE(%s_title, '%s:', ''), "
	     . "%s_namespace = %d\n"
	     . "WHERE %s_title LIKE '%s:%%' AND %s_namespace = %d;\n",
	     $t, $short, $short, $name, $short, $num + $old_ns,
	     $short, $name, $short, $old_ns);
    }
  }
}
?>

It's dangerous to show this code, because someone may come to this page after a Web search and erroneously think this is the way to introduce namespaces in MediaWiki or something. I'll say it again: Don't use this!

Unfortunately, generating SQL statements by-hand is a dirty little secret of database maintenance. I've seen database maintenance done by evaluating SQL commands one-at-time more often than I'd like. Instead of being programmatic and more efficient, "easter egging" methods often introduce typos by either rousing "copy and paste" hell or a "search and replace" hell, and therefore risks typo errors and who knows what else. Writing database maintenance scripts programmaticaly enables you to work as a single transaction and study the results in each iteration. This quality forces one to use a test version of the data and avoid another database faux pas -- working on live databases.

There was a back and forth between Emacs bloggers Jared Dilettante and Ian Eure about whether Emacs users should write Emacs Lisp code for their own purposes or whether working in Emacs Lisp should primarily be in support of existing modes--and with the mode's documentation thoroughly read. At this point, the argument has fizzled out, but its worth pointing out that a manual for SQL mode doesn't seem to exist. SQL mode, like most Emacs modes, is self-documented well. However, this argument is evidence that it could probably use a manual.

As someone who's neither a trained writer nor a good one, I know that writing in one's own spoken language is difficult. However, writing concise documentation about Emacs can be just as important as writing Emacs Lisp code, if not maybe more so. Writing documentation will give you a deeper understanding of how something works and help you learn things you didn't already know. It's also important because the documentation you write will help someone else to learn how to use Emacs, too.

Documenting Emacs also improves your Emacs Lisp skills because you'll likely be reading other hacker's Emacs Lisp code. Most important of all you'll be working on existing code for Emacs rather than making more when it is not entirely necessary to reinvent the wheel.

Don't get me wrong. It's fine to twiddle code. Reinventing the wheel is the basis for higher education and university study and critical to life-long learning. However, writing code and championing your work as a solution only acts to avoid studying and maintaining existing code and risks distracting the Emacs community from progress. In a way, it's almost anti-social.

I know some Emacs hackers think multiple implementations are important, and "I wouldn't have learned as much if I didn't manage my own Emacs package". These arguments are spurious, though. One should be able to earn these same benefits--and more--by joining in on an existing project, rather than forking a new one. It may take more effort but its the right thing to do and is under my column of "best practices".

The GNU General Public License gives the Emacs hacker a lot of freedom, but we all could still use the occasional self-discipline.

Another subtle feature of Emacs that I hope doesn't go away any time soon is an easy way get to the Elisp manual. The key sequence is `C-h r TAB RET'.

Here's how it works. In Emacs 22, a new key binding appeared to access the Emacs manual, `C-h r'. Conveniently the first cross reference in the Emacs manual is "See Emacs Lisp(elisp)". Hitting `TAB' skips you to the first cross reference, then `RET' follows the cross reference.

Interesting discussion on how the REPL buffer of DrScheme is not like Emacs's. DrScheme is the GUI editor for PLT Scheme. PLT was how I learned Scheme in school a decade ago.

The argument pronounced by one of the academics who oversees the software's maintenance is essentially that having an inferior buffer to send and evaluate Scheme expressions is confusing for pedagogical reasons, and even to non-students. Years later, many of PLT followers and supporters want this feature. I recall seeing a PLT researcher who would argue that PLT was real-world, production-ready software.

My favorite is that the author starts it as a letter with the caustic greeting "Dear old Lisper". I've never seen the author speak but have heard of his self-sown reputation as an iconoclast.

I couldn't find a minimum function for my shell scripting, nor a utility on GNU/Linux, so I am using this function.

###
 # min NUM ...
 #
 # Find smallest value of NUMs.
 ##
function min() {
    echo "$@" | tr '[[:space:]]' '\n' \
     | grep -Ee '^-?[[:digit:],]+(.[[:digit:]]*)?$' \
     | sort -n | sed 1q
} ## end min

It supports floating point numbers with decimal notation, but does not support exponential notation or other.

Example:

min 1 0.2 1,023.56 -0 -1.3

Gives: -1.3

A max function is the same thing but needing to change either

• sort -nr
• sed '$p;d'
• sed -n '$p'

PHP has always had a hackish way to support first-class functions by making it possible to assign anonymous functions with create_function and run them with any functions that support callbacks (See array_map).

PHP 5.3 (unreleased as of 2009-03-12) will introduce Lambda functions and closures. I'm not really sure if the implementation is sound or complete, but I'm sure there are enough functional programming language implementations and progammers to make sure its *doing the right thing*. I'm very glad because I was afraid PHP was going to be steered backwards by opinions like this.

See also IBM Developer Works, which had an article on these forthcoming features in What's new in PHP V5.3, Part 2: Closures and lambda functions.

Someone asked recently what the design philosophy of PHP was. I had to say, "PHP was meant to be a server-side include system for HTML documents with a Perl syntax. It's been trying to outrun its heritage ever since."

In the previous installment, I quickly showed how to use GNU Findutils to load file system meta information into a PostgreSQL database. I did so using a comma separated value (CSV) file generated from a tab delimited file. The us of tabs limited the data set to files without newlines or tabs in their names. Here I will show how to load any file name.

The find command can output all possible files by separating the fields in the output with nulls, and each line by double nulls.

Here's the new -printf statement that outputs nulls between records, and two nulls between each line.

$ find / -printf '%i\0%f\0%p\0%h\0%y\0%u\0%U\0%g\0%G\0%M\0%m\0%s\0%b\0%k\0%l\0%n\0%AY-%Am-%Ad %AH:%AM:%AS\0%TY-%Tm-%Td %TH:%TM:%TS\0%CY-%Cm-%Cd %CH:%CM:%CS\0%F\0%D\0\0' > finddb.txt

This Perl scriptlet will convert the output to CSV.

$ perl -mText::CSV_XS -e 'my $csv = Text::CSV_XS->new ({ binary => 1, eol => $/ }); my $n = 21; my @c = (); local $/ = "\0\0";' -ne '$_ .= "\n"; push(@c, split(/\0/)); pop(@c); if ($#c + 1 < $n) {next;} elsif ($#c + 1 > $n) {pop; if ($csv->combine(@c[0 .. $n - 1])) {print $csv->string;} else {printf STDERR $csv->error_input;} @c = @c[$n .. $#_];}' -e 'if (@c > 0) {printf STDERR ("Extra fields at the end\n");}' finddb.txt > finddb.csv

Coincidentally, double-nulls don't help delimit records in the output, since they could be confused as an empty field. So the script above keeps an internal tally of fields in a record, and is hard-coded as 21. Thus, when enough null-delimited fields are read, then the next record is read.

With the resulting CSV file, loading into PostgreSQL is as easy as the following command.

$ psql -c '\copy finddb from STDIN CSV FORCE NOT NULL path, symlink' < finddb.csv

I put the CSV file generation bits together in a Perl script, find-csv.pl. It tries to maintain the consistency between GNU findutils -printf formatting fields, the names of database columns, and the Perl code for generating CSV files.

In a follow-up, I will give more tastings on possible database queries that can be made of this file system information.

Thanks to James Youngman for reading a previous version of this article. This post is the result of just one of his generous comments.

GNU findutils is extremely useful. Unfortunately, the find command doesn't always scale well. For instance, running find on the entire machine can take a long time. And should you learn something that requires modifying your find expression, you have to start the command all over again from the beginning.

The sister program, locate, command helps by being much faster. However, it is missing the expressiveness of find. The locate command should support all the features of find -- save for maybe the -exec expression for security reasons.

Even still, the query syntax for find commands are also not scalable or very user-friendly.

I always wanted to import file system meta information into a database, and use SQL queries to find information about the file system. SQL has its own set of problems, but it would make asking questions about the files on a computer much more worthwhile and maybe even a bit exciting. Very interesting queries could be made, and they would be answered very quickly -- without having to wait.

So, I finally gave it a try. I describe here how I was able to load the results of find into PostgreSQL

The -printf expression of findutils was great for this task. It can generate the information about the file system. And it can have its output formatted to a tab-delimited file.

The following use of the find command makes a text file with entries for every file in your user directory.

$ find ~ -printf '%i\t%f\t%p\t%h\t%y\t%u\t%U\t%g\t%G\t%M\t%m\t%s\t%b\t%k\t%l\t%n\t%AY-%Am-� %AH:%AM:%AS\t%TY-%Tm-%Td %TH:%TM:%TS\t%CY-%Cm-� %CH:%CM:%CS\t%F\t%D\n' > finddb.txt

It's a pretty hideous command. These are all the fields it produces in the output:

• inode
• name
• wholename
• path
• type (file, link, directory, device, ...)
• user (symbolic)
• user_id (number)
• group
• group_id
• perm (e.g. "-rw-rw-r--")
• perm_octal (e.g. "0664")
• bytes
• blocks (512-byte blocks used of disk)
• kblocks (1k-blocks)
• symlink
• links (number of hard links)
• atime (last access time)
• mtime (last modification)
• ctime (last time modified or status changed)
• fstype (e.g. "ext3")
• dev_id (device number)

These represent the header fields in the output. Later, they will represent the table columns in the database.

Unfortunately, the output of find is one-file-per-line and tab-delimited. That means files with newline or tab characters in their names won't cooperate. A suboptimal solution is to just ignore those files on the system. That's easy to do with the find command.

$ find / ! -regex ".*[$(echo -ne '\n\t')].*" -printf '%i\t%f\t%p\t%h\t%y\t%u\t%U\t%g\t%G\t%M\t%m\t%s\t%b\t%k\t%l\t%n\t%AY-%Am-� %AH:%AM:%AS\t%TY-%Tm-%Td %TH:%TM:%TS\t%CY-%Cm-� %CH:%CM:%CS\t%F\t%D\n'

Better yet, complain to the user on standard error (STDERR) every time the find command comes across one of these rare files.

$ find / \( -regex ".*[$(echo -ne '\n\t')].*" -exec sh -c 'echo >&2 "$0": File name has tab or newline' '{}' \; \) -o -printf '%i\t%f\t%p\t%h\t%y\t%u\t%U\t%g\t%G\t%M\t%m\t%s\t%b\t%k\t%l\t%n\t%AY-%Am-� %AH:%AM:%AS\t%TY-%Tm-%Td %TH:%TM:%TS\t%CY-%Cm-� %CH:%CM:%CS\t%F\t%D\n' > finddb.txt

If you want to know if your system has these wickedly named files, run the following locate command.

$ locate -r ".*[$(echo -ne '\n\t')].*"

To convert the file to a comma-separated value (CSV) file, I like to use Perl.

$ perl -mText::CSV_XS -e 'my $csv = Text::CSV_XS->new({ binary => 1, eol => $/ });' -ne 'chomp; split(/\t/); if ($csv->combine(@_)) {print $csv->string;} else {printf STDERR $csv->error_input;}' finddb.txt > finddb.csv

This is a table definition for PostgreSQL that can be loaded with the CSV or tab-delimited text file.

CREATE TABLE finddb (
    inode bigint NOT NULL,
    name text DEFAULT '' NOT NULL,
    wholename text DEFAULT '' NOT NULL,
    PRIMARY KEY (inode, wholename),
    path text DEFAULT '' NOT NULL,
    type character varying(1) NOT NULL,
    "user" text DEFAULT '' NOT NULL,
    user_id integer NOT NULL,
    "group" text DEFAULT '' NOT NULL,
    group_id integer NOT NULL,
    perm character varying(10) DEFAULT '' NOT NULL,
    perm_octal character varying(6) DEFAULT '' NOT NULL,
    bytes bigint DEFAULT 0 NOT NULL,
    blocks bigint DEFAULT 0 NOT NULL,
    kblocks bigint DEFAULT 0 NOT NULL,
    symlink text DEFAULT '' NOT NULL,
    links integer DEFAULT 0 NOT NULL,
    atime timestamp without time zone
          DEFAULT '1970-01-01 00:00:00' NOT NULL,
    mtime timestamp without time zone
          DEFAULT '1970-01-01 00:00:00' NOT NULL,
    ctime timestamp without time zone
          DEFAULT '1970-01-01 00:00:00' NOT NULL,
    fstype text DEFAULT '' NOT NULL,
    dev_id integer NOT NULL
);

Loading the text file into PostgreSQL is as easy as:

$ psql -c '\copy finddb from STDIN' < finddb.txt

for the CSV file:

$ psql -c '\copy finddb from STDIN CSV FORCE NOT NULL path, symlink' < finddb.csv

I did come across a few names on a file system that -- I believe -- Postgres would complain about, because of improperly encoded characters. Postgres on my system expects everything to be UTF-8 encoded. According to James Youngman, the maintainer of GNU findutils,

Character encoding is of course a significant problem. The Unix file system API offers no way to record the character encoding in effect at the time the file is created/renamed, so files on a file system will often have differing encodings.

After the load is completed, here's an example query and a returned row.

=> SELECT * FROM finddb WHERE wholename = '/home/aaronh/.emacs';
-[ RECORD 1 ]--------------------
inode        | 18842194
name         | .emacs
wholename    | /home/aaronh/.emacs
path         | /home/aaronh
type         | f
user         | aaronh
user_id      | 500
group        | aaronh
group_id     | 500
perm         | -rw-rw-r--
perm_octal   | 664
bytes         | 2884
blocks       | 8
kblocks      | 4
symlink      | 
links        | 1
atime        | 2008-11-04 12:26:46
mtime        | 2008-10-24 13:10:24
ctime        | 2008-10-24 13:10:24
fstype       | ext3

The following are some more examples of queries on this database table.

This query finds the 5 largest graphic files that were last modified in 2007, but ignores the auxiliary files of many a version control system.

SELECT wholename, bytes, mtime
FROM finddb
WHERE "type" = 'f' AND "name" ~ '.jpe?g'
      AND path not like '%/.svn/%'
      AND path not like '%/.git/%'
      AND path not like '%/.hg/%'
      AND path not like '%/.bzr/%'
      AND path not like '%/{arch}/%'
      AND path not like E'%/\\_darcs/%'
      AND mtime >= TIMESTAMP '2007-01-01 00:00:00'
      AND mtime <= TIMESTAMP '2007-12-31 23:59:59'
ORDER BY bytes DESC
LIMIT 5;

This query shows every user owning a file on the system, with the the total megabytes used, and with the biggest users first in the list.

SELECT "user", SUM(kblocks) / 1000.0 AS "mbytes"
FROM finddb
GROUP BY "user"
ORDER BY SUM(bytes) DESC;

This query tries to mimic the output of the ls -l / command.

SELECT perm, links, "user", "group", bytes, mtime, name
FROM finddb
WHERE path = '' AND name NOT LIKE '.%' ORDER BY name;
    perm    | links | user | group | bytes |        mtime        |    name    
------------ ------- ------ ------- ------- --------------------- ------------
 drwxr-xr-x |     3 | root | root  |  4096 | 2008-09-22 05:17:44 | backup
 drwxr-xr-x |     2 | root | root  |  4096 | 2008-10-29 18:30:19 | bin
 drwxr-xr-x |     5 | root | root  |  1024 | 2008-10-28 12:43:47 | boot
 drwxr-xr-x |     2 | root | root  |  4096 | 2008-09-10 04:11:52 | cdrom
 drwxr-xr-x |    13 | root | root  |  4460 | 2008-11-12 21:58:07 | dev
 drwxr-xr-x |   117 | root | root  |  8192 | 2008-11-12 21:57:53 | etc
 drwxr-xr-x |     5 | root | root  |  4096 | 2008-11-07 16:20:54 | home
 drwxr-xr-x |    16 | root | root  |  8192 | 2008-10-29 18:29:59 | lib
 drwx------ |     2 | root | root  | 16384 | 2008-09-10 03:05:56 | lost found
 drwxr-xr-x |     2 | root | root  |  4096 | 2008-11-12 21:57:28 | media
 drwxr-xr-x |     2 | root | root  |  4096 | 2008-04-07 17:44:40 | mnt
 drwxr-xr-x |     2 | root | root  |  4096 | 2008-04-07 17:44:40 | opt
 dr-xr-xr-x |   107 | root | root  |     0 | 2008-11-12 21:55:52 | proc
 drwxr-x--- |     6 | root | root  |  4096 | 2008-11-12 20:02:29 | root
 drwxr-xr-x |     2 | root | root  |  8192 | 2008-10-29 18:30:18 | sbin
 drwxr-xr-x |     7 | root | root  |     0 | 2008-11-12 21:55:52 | selinux
 drwxr-xr-x |     2 | root | root  |  4096 | 2008-04-07 17:44:40 | srv
 drwxr-xr-x |    11 | root | root  |     0 | 2008-11-12 21:55:52 | sys
 drwxrwxrwt |    74 | root | root  |  4096 | 2008-11-13 00:56:28 | tmp
 drwxr-xr-x |    13 | root | root  |  4096 | 2008-09-10 03:15:15 | usr
 drwxr-xr-x |    21 | root | root  |  4096 | 2008-09-24 06:53:23 | var

Sending queries against the data is loads of fun, but it really needs some improvements to match the strength of findutils matching expressions -- for example, the permissions matching rules and the -empty predicate. Some new tables with alternative perspectives on the data could accommodate better queries.

In a follow-up, I will present on how to handle all possible file names by using a null-delimited file rather than a tab-delimited one. There may be a piece on loading into MySQL. And in the last piece, I'll give more tastings on possible queries can be made of this file system information and provide the script that helps me manage the database loads from find.

The Samsung S2 (YP-S2) is a new flash memory music player that advertises on the box to play "Ogg". At $40, it's one of the most inexpensive Ogg Vorbis playing devices yet. The 1GB media players lacks a touch screen or any type of screen for that matter. Its only interface are universally recognized playback buttons that include volume control. It has a complicated "Smart Button" that you either press or hold while in different modes for different meanings, including changing either the sound engine, playback mode, turning off the LED light, changing the play list, and making new play lists. It's "smart" because you need to be smart to use it.


(Fits in the palm of your hand.)

Marketed as "the pebble", it is small and light. Pictures on the Web don't really communicate its compactness until you experience it in-person. I assumed it was the size of a hockey puck, but it fits the middle of your hand. For better photos of the S2, see the review of the S2 at anything but ipod.com. The sound quality is surprisingly good. Samsung is trying to compete with Apple's iPod Shuffle by price-point, but is a better alternative since it has out-of-the-box Ogg Vorbis support. The iPod shuffle is not compared here.

The S2 comes with a pair of earphones, a USB adapter and a tiny installation CD. It requires being charged by USB when you take it out of the box. The manual suggests charging for 2 hours. The subtle and translucent LED light turns from red to green when the battery is fully charged. While waiting for the S2 to charge, you should be transferring your files (see below). The transfer rates are slow for USB (less than 1MB per second), but you can fill the S2 with 25 to 30 albums of music in 15 minutes. (Copying from the drive is at speeds around 4MB per second.)

The earphones have a special adapter on the plug to clasp the pebble tightly. Combined with a nylon neck strap on the headphones, the S2 hangs on your chest quite comfortably. The lanyard provides another nickname for the device, "pendant". There's no clothing clip for the S2, but securing it would only be necessary if you were bouncing during aerobic activity.

The USB adapter for the S2 plugs into the same hole as the headphones. After connecting the USB adapter into a GNU/Linux machine, it will be mounted as USB mass storage device called "S2". There are 3 folders on the S2 -- music, playlist and system. I removed the demo mp3 files in the music folder and the one playlist file. I copied my collection of Ogg files from the command-line with the command cp -a ~/music/* /mnt/S2/Music.

Its not clear if any of the default folders of the S2 are necessary. They don't take up much space. Should removing them disable the player, there's a system reset button available in a pin hole that advertises to re-initialize the system when the S2 "won't play music, or isn't recognized by your computer when you connect it."

Features like upgrading the firmware and programming playlists are available through the proprietary software that only runs on Microsoft Windows. The device also enforces Digital Restrictions Management -- by not playing them. Fortunately, you can avoid these drawbacks by using Ogg Vorbis, and not downloading DRM-encumbered files. You can manage the playlists from the player itself, but the button-pressing to do this is gruesome and playlists are limited to only 30 songs, anyway. Since there isn't a way to navigate albums (folders), a playlist with a song from each album on the player would be useful, however. According to the Portable players page at Xiph there are no techniques to over ride the S2's firmware, yet.

The battery is advertised to last 13 hours, though that's hard to believe. The ability to disable the LED light (see below) would probably not lengthen the battery's charge. Turning down the volume would probably work better. The battery does not appear to be replaceable, either. The battery is likely covered by Samsung's one year warranty, however.

The S2's LED indicator is the only visual display -- if you can call it that. There is a beeping system as well. In addition to the battery status (see above), the light is blue when the player is turned on. The LED is blue during regular playback, and red if playing a playlist. If in shuffle mode, it cycles between all the colors (including green). It signals red when the power is low, too. Should you get sick of the light show, you can disable it by pausing the music, then holding the Smart button. Doing the same again turns it back on.


(The S2's LED is glowing blue)

If you don't have a USB adapter for your car stereo, then you'll need to purchase either an FM transmitter or get a standard 3.5 mm (1/8") audio cable (TRS). Unfortunately, there's no way to charge the device while it plays, but it could be charged in a USB car adapter that plugs into the cigarette lighter.

Besides the S2's proprietary software support, the other major design flaw is the location of the headphone jack. The instinct is to hold the headphone wire at the bottom of one's hand and use the thumb to hit buttons. Instead, the jack is at the top of the device. So when you grab the S2 upside down, adjust the volume slowly. You may be cranking it up when you mean to turn it down. Although not person-oriented, this configuration is natural for when the device is plugged into something else, like a stereo.

Samsung will likely be releasing a 2GB version of the player. This might be handy, but you'll run up against battery capacity before playing the first gigabyte of songs. I'd prefer owning two 1GB for that reason. With two you get more battery, and on a really long car trip you could have one charging while the other is playing.

Samsung should do more to support free software operating systems with the S2. But, this is not surprise since Samsung is a deal-maker and collaborator with Microsoft against GNU/Linux. Regardless of their hostility, the player is a great way to support the Play Ogg! movement.