Saturday, November 18, 2017

Making Native Language Support (NLS) and gettext Optional

I've decided to write this blog post after noticing how many people were asking me to make gettext optional when building fswatch. At the beginning I actually thought it was optional. And I never dug any deeper into the matter because, apparently, everybody experiencing this problem was building fswatch from the repository sources instead of building it from a release tar-ball. I would typically answer to build the release tar-ball instead and I would hear no more complaints.

Making gettext optional is easy, and should be a best practice when writing a piece of software. However, I've seen so many questions in the Internet about this topic that I've decided to write a blog post where I would outline the reasons why you should do it and how you should do it using the GNU Autotools.

So, What's the Issue With Building From Repository Sources?

Actually there's no issue. But probably many people asking that question don't know how the Autotools work. Autoconf is a tool whose goal is producing scripts (the ubiquitous configure script) that configure the source code before building it to adapt it to the characteristics of the current system. In a typical flow the user does nothing but running configure. The script will run and will test for the presence of all the features requested by the author. The presence of a feature, or its lack thereof, can then be used to parametrise Makefiles, or more generally, be used to conditionally drive the execution of other tests or scripts.

The Autotools are not required by end users. The scripts generated by the Autotools are fully independent from them. These script are then bundled with the sources (in a distribution) which is often called a release tar-ball.

As you can see, the goal of the Autotools is making users' life simple, not maintainers'. In fact, maintainers need to have the Autotools components they use in order to generate a distribution. These are the sources you typically find in a package repository such as fswatch. That's why you don't find any configure script in the source repository: it's created when a release is created and a distribution is packaged.

Perhaps not surprisingly, GNU gettext is one of the components software authors may use conditionally. configure would then check the build environment to detect the availability of all the required gettext components and enable Native Language Support (NLS) if all checks succeed.

What's Special About Gettext?

This Autotools overview doesn't explain, though, what's the issue about gettext. If it's just one of the many configuration checks performed by configure, what's the relationship with the Autotools and the repository sources? The answer is that gettext provides a tight integration with the Autotools, in the form of ready to use Autoconf and Automake macros to support the gettext operations during a normal workflow, such as:
  • Extracting strings from sources.
  • Creating template files (POT files).
  • Creating translation files (PO files).
  • Updating template files.
  • Merging changes into translation files.
  • Compiling translation files into message catalogs.
  • Shipping required files in a distribution.
  • Installing message catalogs.

It goes without saying that the maintainers do need gettext installed in their machines in order to be able to use these macros, to create the configuration scripts and to create a package distribution.

Making gettext Optional

To make gettext optional, we have to perform at least these operations in the Autoconf configuration file (configure.ac):
  • Use the AM_GNU_GETTEXT macro to check whether NLS should be used. If it should be used, the USE_NLS variable would be set to yes.
  • Create an Automake conditional to propagate this information to the Makefiles, perhaps using something in the lines of:
AM_GNU_GETTEXT([external])
AM_GNU_GETTEXT_VERSION([0.19.4])
AM_CONDITIONAL([USE_NLS], [test "x${USE_NLS}" = "xyes"])

In the Makefiles, we have to leverage these information to:
  • Conditionally distribute NLS-related files such as ABOUT-NLS:
if USE_NLS
  dist_doc_DATA += ABOUT-NLS
endif
  • Conditionally execute NLS-related targets. What I typically do is segregating all NLS-related stuff in a directory (po) which is conditionally processed by the Makefile:
if USE_NLS
  PO_SUBDIR = po
endif

SUBDIRS = other_paths $(PO_SUBDIR)

All the other Makefile fragments required to use gettext, such as defining the LOCALEDIR to load catalog files and linking against libintl, do not need to be modified. Even if always adding @LTLIBINTL@ to LDADD may seem odd at first, the Autoconf output variable LTLIBINTL is documented to expand to an empty string when NLS is not being used.

# Prepare gettext-related symbols used by programs
AM_CPPFLAGS += -DLOCALEDIR=\"$(localedir)\"

# Link program against libintl if gettext is being used
prog_LDADD += @LTLIBINTL@

Conclusion

This is all that's required to make gettext optional in your Autotools package. With these few changes, users will now be able to let configure automatically determine whether to enable or disable NLS, and even disabling it using the --disable-nls option:

$ ./configure --disable-nls




Saturday, May 2, 2015

A Shell Script to Delete Old Time Machine Backups and Free Disk Space

In another post I described the procedure to manually delete Time Machine snapshots and optionally shrink the disk image (in case a sparse bundle is being used) in order to free up disk space.

I have written a shell script, tm-cleanup.sh, to automate the backup deletion. The script accepts an optional argument to specify the maximum backup age in days: older backups are deleted (by default, backups older than 30 days are deleted).

The latest version of the script can be obtained by cloning its GitHub repository and, optionally, run its installation script to create symbolic links to it into /usr/local/bin:

$ sudo make install


Friday, October 17, 2014

Installing Drupal on FreeBSD

Drupal ports have been available on FreeBSD since quite a long time, and binary packages can be installed very quickly. However, manual setup is required to connect Drupal to the database and have Apache serve the Drupal website. In this post I'll describe the setup procedure of Drupal 7 on FreeBSD 10.0. The process will not be very different if different versions of Drupal or FreeBSD are used.

Installing Drupal

Drupal ports are available on FreeBSD and, in fact, multiple versions are available:

# pkg search drupal
drupal6-6.31
[...snip...]
drupal7-7.31
[...snip...]

Unless there is a compelling reason not to do so, install the latest one:

# pkg install drupal7

To successfully run Drupal, you need:
  • The Apache HTTP Server.
  • PHP.
  • A supported database server (PostgreSQL or MySQL).

The Drupal port, however, does not enforce these dependencies, so that you have to satisfy them manually.

Installing the Apache HTTP Server

Unless there is a compelling reason not to do so, install the latest available Apache port (apache24 at the time of writing):

# pkg install apache24

Once the port is installed, enable the corresponding service adding the following line to /etc/rc.conf:

apache24_enable="YES"


Installing the Database

Drupal supports both PostgreSQL and MySQL but the Drupal port does not install any, by default, although it installs the MySQL client utilities. In this post MySQL will be used but if you prefer using PostgreSQL instead, just skip this section and read this article instead.

Since the Drupal port by default defines the MYSQL option, when you install the binary package using pkg you'll also get a MySQL client port, such as what I got at the time of writing:

mysql55-client-5.5.40

As a consequence, you have to install the matching mysqlXX-server port:

# pkg install mysql55-server-5.5.40

If you try to install a different version (at the time of writing mysql56 is available), you may be requested to remove Drupal itself because of the inter-dependencies between the client and server MySQL packages.

Once MySQL is installed, enable the corresponding service adding the following line to /etc/rc.conf:

mysql_enable="YES"


Installing PHP

The installation of PHP is taken care of by the Drupal port. However, the PHP module for the Apache HTTP Server is not installed and must be installed manually. Make sure you install the PHP module that corresponds with the PHP version installed by the Drupal port. At the time of writing, the following modules are available:

# pkg search mod_php
mod_php5-5.4.33_1,1
mod_php55-5.5.17_1
mod_php56-5.6.1

Since the port depends on php5, then mod_php5-5.4.33_1,1 should be installed:

# pkg install mod_php5-5.4.33_1,1

The port takes care of modifying the Apache HTTP Server configuration file so that the PHP module is loaded. If you did not install the packages in order suggested by this post, then you may have lost that piece of configuration. In any case, make sure a line similar to the following is present in /usr/local/etc/apache24/httpd.conf:

LoadModule php5_module        libexec/apache24/libphp5.so


Installing drush

drush is an optional package offering an amazingly good command line interface to perform many Drupal-related management tasks: drush could even be used to install Drupal, but this topic is not covered by this post, since I prefer relying on a port tested specifically on FreeBSD. However, once you have tested that a specific release works correctly, you may find drush very useful to streamline your installations. If you are interested in using drush, you will find plenty of good information on the Internet. To install drush, the following command may be used:

# pkg install drush


Creating a Database for Drupal

A database for Drupal must be created in the DB server installed in the previous section. The MySQL port sets no password for the root user connecting from localhost; for this reason, setting its password is recommended (in bold the text input by the user):

# mysql -u root
Welcome to the MySQL monitor.  Commands end with ; or \g.
[...snip...]
mysql> set password for root@localhost=PASSWORD('your-password')
mysql> exit
Bye

Once the password is set, you can try to reconnect (in italic bold the variables whose name should be changed with your values of choice):

# mysql -u root -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
[...snip...]
mysql> create database drupal_database_name;
mysql> create user 'drupal_user'@'localhost' identified by 'password';
mysql> grant all privileges on drupal_database_name.* to 'drupal_user'@'localhost' with grant option;
mysql> flush privileges;


Configuring the Apache HTTP Server

Now that everything is in place, we can configure the web server so that it starts serving the Drupal web application. The tasks to perform are the following
  • Configuring the required modules.
  • Configuring a virtual host to serve Drupal.
  • Configuring a MIME/type for PHP.

The modules required to run Drupal are mod_rewrite and the PHP module. The latter was configured automatically by the PHP module port, and the latter can be configured uncommenting the following line from /usr/local/etc/apache24/httpd.conf:

LoadModule rewrite_module libexec/apache24/mod_rewrite.so

The cleanest way to segregate the Drupal configuration is creating a virtual host for it. An additional advantage of this approach is that Drupal will be served from the root path (/) and you won't need to use any rewrite rule to achieve the same result. Assuming the host name and the port where Drupal will be published is drupal.host.name:80, then create a file in /usr/local/etc/apache24/Includes named drupal.conf and define the skeleton of the virtual host:

<VirtualHost *:80>
  ServerName drupal.host.name

  # Drupal directory configuration placeholder
  
  ErrorLog ${APACHE_LOG_DIR}/drupal-error.log
  LogLevel warn
  CustomLog ${APACHE_LOG_DIR}/drupal-access.log combined
</VirtualHost>

In the default configuration of Apache in FreeBSD, any .conf file in this directory is included automatically, so that no additional code is required to add the virtual host to the web server configuration. In this fragment I've used an environment variable (${APACHE_LOG_DIR}) to separate some server configuration variables that could be reused in external scripts. To define environment variables, a .env file must be created in /usr/local/etc/apache24/envvars.d such as:

The Drupal directory fragment defines the DocumentRoot of the virtual host and some of the required options:

DocumentRoot /usr/local/www/drupal7
<Directory "/usr/local/www/drupal7">
  Options Indexes FollowSymLinks
  AllowOverride All
  Require all granted
</Directory>

The option AllowOverride set to All is required so that .htaccess files shipped with Drupal are taken into account by the Apache HTTP Server. In this fragment, the path of the Drupal installation directory of the FreeBSD port is used. If you installed Drupal using alternative methods (such as drush), update the path accordingly.

The complete virtual host configuration file is:

Finally, the Apache HTTP Server must be instructed to execute the PHP code contained in PHP pages and to do so we need to add a MIME/type for them adding the following line in httpd.conf:

<IfModule mime_module>

  # Content has been trimmed

  # Add MIME type for PHP
  AddType application/x-httpd-php .php

</IfModule>

Once all the settings are in place, Apache can be restarted and you can point your browser to http://drupal.host.name/ where the Drupal installation wizard will welcome you and will require you to input the database configuration and other Drupal website settings.

To restart the Apache HTTP Server, the following command can be used:

# service apache24 restart

Configuring Drupal behind a Proxy Server

Machines connecting to enterprise network often are not connected directly to the Internet but require the use of a web proxy server instead. Drupal can be configured to use a web proxy server by setting the following variables in ${DRUPAL_HOME}/sites/default/settings.php. If this file does not exist, copy the file default.settings.php (shipped with Drupal) into settings.php. The configuration variable that enable proxy support are the following:

Depending on your proxy settings, different values may be used.

Beware that although Drupal itself (Core) supports a proxy, many third-party modules still do not. One notable exception at the time of writing is the reCaptcha module which will not work without a direct Internet connection.

Setting Up Clean URLs

Last but not least, clean URLs support may be enabled. Drupal performs a sanity check and will not allow you to enable the Clean URLs feature if the test does not pass. However, I have found plenty of false negatives when running Drupal 7 on FreeBSD: if the Clean URL test fails in your installation, try checking if clean URLs are working and use the workaround described in the official Drupal documentation and forcibly enable Clean URLs.

Saturday, September 27, 2014

fswatch 1.4.5 - Release Notes

fswatch 1.4.5 has been released.

New features and bug fixes added since v. 1.4.0 are:

  • Add custom record formats.
  • Localize fswatch and libfswatch using GNU gettext.
  • Add Italian (it) localization.
  • Add Spanish (es) localization.
  • Fix Makefile.am because of broken link when DESTDIR installs are performed.
  • Fix bug in fswatch-run wrapper script for ZSH which caused last argument not to be split when passed to xargs.
  • Add batch marker feature to delimit the boundaries of a batch of events.
  • Add Texinfo documentation.
  • libfswatch API is now versioned.
  • Improved Autoconf checks.
  • The inotify monitor now waits for events and honours the latency settings.
  • Automaticaly generate the ChangeLog using Git.
  • Update autogen.sh to honour some commonly used environment variables.
  • The inotify monitor now provides the same functionality provided by all the other monitors.
  • Recursive directory monitoring is now implemented.
  • Version and revision is now determined dynamically from Git by ancillary scripts invoked by the GNU Build System.
  • fswatch does not compile on OS X < 10.9 because some required C++11 classes are not supported by the C++ runtime.
  • fswatch does not compile on OS X < 10.9 because some required C++11 classes are not supported by the C++ runtime.
  • The libfswatch library has been added with bindings for C and C++.
  • fswatch let users specify the monitor to use by name.

Thursday, August 21, 2014

fswatch 1.4.0 - Release Notes

fswatch 1.4.0 has been released bringing few but important updates:
  • Monitor implementations are found at runtime so no more hardcoded options to choose a specific monitor (the plugging mechanism still needs improvements).
  • The fswatch engine has been moved to a separate library called libfswatch with bindings for C and C++. If you need to use the functionality of fswatch from your program, now it's very easy to do. The library and its headers are distributed in the same fswatch bundle.
  • On OS X fswatch can now be installed from either MacPorts and Homebrew (see README for more information).

I hope you enjoy using fswatch as much as I enjoy writing it.