|
|
Source Code UNIX on the PC
By Bob Gray
Bob Gray is co-founder of Boulder Labs, a digital video company.
Designing architectures for performance has been his focus since he
built an image processing system on UNIX in the late 1970s. He has a
PhD in computer science from the University of Colorado.
Application Software: Ports and PackagesThree times in the period of a week I had to download the latest source files for publicly available applications (gnuplot, glimpse, psutils) and compile them for my six-year-old SGI Indigo. I needed current versions because of new features or bug fixes. The time spent for this maintenance was two hours, but it is a never-ending, thankless, nonbillable chore. I have about 175MB of frequently used binaries under /usr/local, including bash, gawk, emacs, ical, exmh, Tcl/Tk/Expect, ssh, groff,pgp, and tgif. Facilities to minimize the effort of keeping up to date are desirable. The Source Code UNIX systems offer help in this area. On my networked Source Code UNIX Pentium processor, I also rely on these freely distributed programs, but there is much less work in keeping them up to date. On the FreeBSD system, it's simply a matter of going to the appropriate subdirectory in /usr/ports and typing "make." If a current CD is mounted, the sources for the target program will be copied; then the binaries will be built. If the CD is out of date or missing, sources for the program will be fetched from the Internet and a binary built. Information about current software versions is in the hierarchy /usr/ports. The make program looks in /usr/ports to figure out what is the current version of an application. Then make looks for matching source code on the CD-ROM. If current source code is not available on the CD-ROM, make will attempt to download source code from the Internet. Then the binaries are built and installed. Finally, an entry is made in a simple database tracking what software is installed so that it can later be updated or removed (pkg_delete). Of course, /usr/ports must be keep very current for this to work well, but the small /usr/ports tree is easy to keep up to date using cvsup, a recent CD-ROM, or ftp. This article discusses freely available application software for UNIX platforms, how to get it, and how to install and maintain it. My first article (online at <www.boulderlabs.com>) gives many reasons for running Source Code UNIX. Having access to a couple of thousand publicly available applications is one good reason. And access to the source code for those applications is a second good reason. Whether you are in a big office with a platoon of system administrators or in a solo home office, it takes a lot of work to keep software up to date. Anything that helps is worth looking at. Folks working for the advancement of Linux, FreeBSD, NetBSD, and OpenBSD collectively make installing these applications easier for you. They monitor the development and release cycles of the applications, and they work to make installation as automatic as possible. Source Code UNIX distribution CD-ROMS have many of the applications precompiled and ready to go. I'll talk about the mechanism used to distribute and install these applications for FreeBSD, NetBSD, and OpenBSD. For Linux, the discussion depends on the particular distributor of the CD-ROM set. One of the most popular is Red Hat Linux with its rpm files. FreeBSD uses the term "port" to refer to the source code for an application. The tree /usr/ports starts out as a small hierarchy containing for each of the more than 1,600 application, a makefile, a checksum, a version number, a description and possibly some patch files. The application source code initially does not reside under this tree. You can think of the Ports Collection as an infrastructure for building any of the 1,600 applications. In FreeBSD, a package is a tar-ball that contains binaries and affiliated files to run an application. A package is similar to a Linux .rpm file or an SGI inst file. (Packages are created by typing "make package" in an application directory of the /usr/ports directory.) Packages are just conveniences so that the end-user doesn't have to spend time compiling. The FreeBSD CD-ROM set has both ports and packages on it. This article addresses freely redistributed software. Here we do not discuss the growing body of third-party, commercial applications for our platform. Big-name vendors porting to Source Code UNIX include Oracle, Informix (databases), Corel (WordPerfect, spreadsheet, etc.), Electronic CAD/CAM, and others. There are alternatives to running Microsoft Office! Check out Applixware <www.applix.com> and StarOffice <www.caldera.com/products/staroffice> -- both are complete office suites available for Linux. See <www.linux.org/news/index.html> for articles on the growing body of commercial software for the UNIX platform. Linux has the largest number of installed systems (maybe 7 million); the vendors usually port to this Source Code UNIX first. But the 1.5 million FreeBSD installations can immediately take advantage of these binary releases using Binary Linux Emulation. See <www.linux.org/apps/index.html> for a list of commercial software and see <www.freebsd.org/handbook/handbook.html> for more information on Linux Emulation. The Ports Collection You might wonder why there is a Ports Collection instead of just putting binaries into the distribution. There is just far too much publicly available software. Most people don't have room on their disks for all of it. Much of it is specialized and not of general interest. Then there are controversial (religious) issues about editors, shells, Web browsers, etc. By making Ports optional, the core distribution is kept modestly small and people can customize (add to) their systems as desired. Under FreeBSD, there are two basic mechanisms for loading optional application software: Ports and Packages. Packages are binary distributions, ready to install. They were created from source code fetched by the ports mechanism. To easily deal with these packages we have
pkg_add -- a utility for installing software package distributions For Linux, the equivalent system is called RPM or the Red Hat Package Manager. See <rufus.w3.org/linux/RPM> for details. The Ports Collection is an elegant example of distributed software cooperation. The authors of the software make their releases available on FTP sites. A few "ports maintainers" monitor the release cycles of this software and modify the /usr/ports hierarchy to reflect current versions. These guys make sure that the virgin software releases will compile cleanly on the FreeBSD system. If necessary, they create some patches in /usr/ports that will be applied before compilation is attempted. This way thousands of end-users don't have to customize the same software over and over to run in their environments. To build and install a port, change to the appropriate subdirectory of /usr/ports and type:
make -- compile the application NetBSD and OpenBSD use very similar mechanisms leveraged off of the FreeBSD Ports Collection. In general, once some application software builds on one of the BSDs, it builds on all of them. The Downside of Ports I would be remiss if I didn't talk about some of the negatives associated with the ports. My colleague, Mike Durian, who has a love/hate relationship with port, is keeping me on an even keel in this article. The application software comes from all corners of the world. Everyone's development environment is different. Whereas SUN, SGI, Apple, and others have strong control over the hardware and software that gets shipped, Source Code UNIX systems have cobbled together hardware and mix and match software from Timbuktu. This set of circumstances makes it impossible for everything to work automatically all of the time. With the Ports Collection, you don't get to choose where things get installed. There are a couple of environment parameters that control some general layout formats, but you really can't fine-tune things. (We can see a need for some kind of a software registry database.) Sometimes one port depends on another. SUN, SGI, and others have tools that build dependency graphs to ensure that things remain consistent. When conflicts arise, you are given choices on how to resolve them. The ports mechanism is not as sophisticated and consequently can get into more trouble. There are assumed locations (generally /usr/local) for certain files and problems develop when various pieces of software make different assumptions. A good example of this is the Tcl/Tk package. If you get and install Tcl/Tk by hand then the chances are you will not be able to use any other packages from the ports collection that need Tcl/Tk. There is a rule in the bsd.*port*.mk makefile template that checks for the default installation location for Tcl/Tk and complains if it is found installed there. The Ports Collection wants it installed according to different rules as defined in the ports makefile. Now you can debate whose installation location is best, but that's not the point. The point is the ports package installed it differently and incompatibly from the standard package. Because you've tried to make things easier for yourself by using the Ports Collection, you've pretty much locked yourself into always using it. It is difficult to mix and match using the Ports Collection with installing and porting things by hand. Another downside of the Ports Collection is that tons of assumptions are being made for you about the behavior of your software. For example, the MH (message handling) software is difficult and time consuming to install by hand. The Ports Collection makes it much easier. However, during the exercise of hand porting, you learn about the configuration issues. When you use the ports version, you've got to hope the maintainer built it with the features you want and need. You should watch the build as it happens. Often the build will display for you to read a file that has important information on something you need to configure by hand before running the application. It is all too easy to miss these messages in the stream of compile commands. Mike also points out the less people are forced to port software themselves, the less they learn about other good styles and practices. He feels fortunate that he had to struggle with other people's software before ports were available. He feels that he is now a much more rounded, experienced programmer. Bob's "Can't Do Without" Software List I've come to depend on a lot of publicly available software. I carry this repertoire wherever I go. All of these are conveniently part of the CD-ROM distribution and the Ports Collection. My shell is bash. Its filename completions and command line editing motivated me to switch from csh years ago. Because of patent issues, the once public compress has fallen out of favor and gzip has taken its place. A nice bonus is that gzip attains significantly better compression. GNU's tar smoothly works with gzip to give you archiving and compression with one command (i.e., tar -z). Written communication is essential for most of us. These tools help me be more effective: look, dict, and ispell help me with spelling daily. Even though it's not my primary editor, emacs is one of my relied upon tools. I save lots of time with its keyboard macro facility. You "record" a set of steps to do something to one line (for example) and then "replay" the macro thousands of times to get the whole file. groff gives me typesetting software; Postscript tools allow me to manipulate it: ghostview, gs (Note, Ghostscript [gs] also reads PDF files), nenscript, psselect.ps2a lets me get ASCII back. To handle the mail, exmh makes dealing with volumes of messages as painless as possible. The winning feature for me is that messages are "presorted" into various in-boxes. Metamail (MIME), glimpse (searching), and pgp (encryption) are built in. The need for security increases as our computers become more networked. PGP is a great way to communicate securely with others using public-key encryption. When you need to securely connect to a host across the Internet, ssh is the ticket. You can log into your home machine over an insecure network using one-time passwords à la skey. The sudo command gives you one time superuser privileges and logs your activity. My home directory is 400Mb in size now. It's easy to find things with the indexing tool glimpse. Its companion agrep is a fast search tool for "approximate" searches when precise regular expression searches won't find it. wget is a batch retrieval program for fetching files from the World Wide Web using HTTP and FTP. You don't have to sit around clicking buttons to retrieve a hierarchy over a slow link.
When I need to plot data, I turn to gnuplot. I frequently use
xv to view/convert most kinds of images. pbmplus
(Portable BitMaps) is the "universal" image conversion I handle my spreadsheet needs with sc. expect automates dealing with interactive programs far beyond what is possible with shell scripts. gawk has the features of the updated Awk programs. I find it a great tool for manipulating tables. I schedule my time with ical. It has alarm features and repeat dates. tgif is my drawing tool. Now that it is hypertext linked, you can make nice interactive presentations and tutorials. Most of us need tools like Perl, rcs, and cvs every day. A Tour of the Ports Collection Below I highlight pieces of the Ports Collection -- attempting to give a feeling of what is available in the broadest terms. You will need to go to the source to check on the details of these applications. You can use the make search= facility to help find applications. (See the handbook). Also glance at the README documents in the sub-directories of /usr/ports:
<www.freebsd.org/handbook/ports.html>
Astronomy
Audio and Multimedia
Benchmarks
Biology
CAD Tools
Converters
Databases
Development of Software
Editors
Emulators
Games
Graphics
Foreign Language Packages
Languages
Mail
Math
Misc
Net
News
Print
Security
Shells
Sysutils
Textproc
WWW
X11
Sources and ReviewersI'd like to thank Mike Durian for helping me keep this article well balanced. Tom Poindexter and Jordan Hubbard also helped with reviewing.
|
 Need help? Use our Contacts page.
First posted: 13th November 1998 jr Last changed: 29 December 1998 jr |
|
Issue index