Executive Summary of the System Administrator Occupational Analysis

Purpose of Project
The System Administrators Guild (SAGE) works to promote the interests of system administrators, particularly with regard to their professional development. One strategy that SAGE is interested in pursuing is the development of a certification program. SAGE is also conducting activities in support of the education and training of system administrators. The Human Resources Research Organization (HumRRO) worked with SAGE throughout 1999 to conduct an occupational/job analysis that could be used to support both of these areas of interest. This is a summary of their report.

Overview of Approach
The occupational-analysis process used available materials, interviews, workshops, and a survey of system administrators to describe the core requirements of the occupation. First, lists of the tasks required for the job and the knowledge areas, skills, and abilities (KSAs) required to perform those tasks were developed. The lists were reviewed and revised by system administrators on existing SAGE committees and those who volunteered to participate in a series of workshops. The lists were incorporated into a Web-based survey that was administered to system administrators. Survey respondents indicated which tasks and KSAs were applicable to their jobs and, for those that were relevant, rated their relative importance to overall job perfor-mance. Respondents were also asked to answer several questions related to their background (e.g., job tenure) and to issues related to certification and professional-development activities. Analysis of the survey data was used to identify the most important tasks and KSAs across the system-administrator occupation and at various skill levels.

Three advisory groups assisted in the planning and implementation of various stages of the project. The SAGE Certification Subcommittee, with the assistance of SAGE staff, was HumRRO's primary source of guidance and input for the project. This group included four subject-matter experts. Additional support was provided by the SAGE Executive Committee and the SAGE Certification Advisory Committee, a group of approximately 48 professionals who had offered to assist in the occupational-analysis project.

Results
The survey was available on the USENIX Web site from October 17 to November 17, 1999. When the survey-administration window was closed, a total of 1,217 surveys had been received.

Although 1,217 survey responses were received, not all were suitable for inclusion in the analyses. For example, system administration was not the primary activity for 177 (14.6%) respondents. There were 30 (2.5%) respondents who had less than one year of experience as a system administrator. Because these 198 (16.3%) respondents had too little knowledge of system administration to make informed ratings of the importance of tasks and KSAs, they were excluded from all analyses. Deletion of these responses resulted in a final sample of 1,018 respondents.

Currently, there is no relatively comprehensive source of information about system administrators (e.g., who they are, how many there are). Therefore, the survey was not sent to specific potential respondents, but, rather, it was announced in locations system administrators would likely visit. Because of this, it is not possible to calculate a response rate (defined as the number of surveys returned divided by the number of surveys distributed). Nor is it possible to evaluate the representativeness of the obtained sample. That is, there is no way to tell whether, for example, the survey sample over- or underrepresents system administrators working with very large numbers of computers or particular operating systems.

Table 1. Sample Size and Years of Experience by System Administrator Level

	Years of Experience in System Administration
Status	Total	1—2	3—5	6—10	10 or more
Beginner	127	97	24	5	1
Intermediate	471	0	250	153	68
Senior	420	5	75	161	179
TOTAL	1018	102	349	319	248

Table 1 shows the final sample sizes for each of the newly defined levels of system administrators, along with the years of experience reported by respondents at each level.

Table 2. Frequencies (Expressed as Percentages) of Level by Primary Setting

Primary Setting			Level
		Total	Beginner	Intermediate	Senior
Commercial		41.4	46.5	41.2	40.0
Academic/Research		23.4	28.3	24.8	20.2
Consultant		10.4	7.1	9.8	12.1
Government		8.1	8.7	8.3	7.6
Financial		7.3	0.8	6.6	10.0
ISP		6.8	5.5	6.2	7.9
Military		2.8	3.1	3.2	2.1
TOTAL		100.0	100.0	100.0	100.0

Table 2 indicates that the largest proportion of the total sample (41%) work in a commercial setting, following by 23% who work in an academic or research setting.

Table 3. Frequencies (Expressed as Percentages) of Level by Major Job Responsibilities

Responsibilities				Level
	Total		Beginner		Intermediate		Senior
General Sys Admin		96.5		96.1		97.7		95.2
Network Admin/Mgt		54.2		50.4		49.7		60.5
Security		44.8		35.4		40.1		52.9
Help Desk		30.1		34.7		29.9		28.8
Support Engineer		29.1		22.8		2 9.1		31.0
Development/Programming		24.4		18.9		21.0		29.8
Web Hosting		24.1		18.1		22 .9		27.1
Database Admin/Mgt		21.8		19.7		19.5		25.0
Facilities Mgt		13.6		10.2		11.9		16.4
Sales		1.3		1.6		.6		1.9
Architecture		0.8		0.0		0.0		1.9
Project Mgt		0.4		0.0		0.2		0.7
Webmaster		0.2		0.8		0.0		0.2
Other		3.1		1.6		1.9		5.0

Note: Each value represents the percentage of respondents who indicated that they perform the responsibility listed in that row. Most respondents marked more than one responsibility.

Table 3 shows the major job responsibilities reported by the survey respondents. There were some interesting differences here among the three experience levels. For example, responsibilities related to security, development/programming, and Web hosting are more often assumed by senior administrators than by either intermediate or junior administrators.

Additional information about the 1,018 respondents indicates that almost 79% work at a high-complexity site. A high-complexity site is defined as one that supports 100 or more computers, 500 or more users, and/or 5 or more operating systems. Again, the more experienced administrators tend to work at the more complex sites.

Respondents marked which of several operating systems they use. Most respondents marked multiple systems. A large majority (76%) of the respondents report using Solaris and a smaller proportion (55%) use Linux. Slightly more than half (53%) of the respondents report using Windows NT. Almost 75% of the survey respondents work in the U.S.; the second-best-represented country was Australia, with 8.5%. The sample was 87.2% male and 88.2% white.

The relative importance of tasks and KSAs was analyzed and ranked. The resulting ranking can be used as a blueprint for education and testing. For example, only those tasks and KSAs that ranked more important than others would be the focus of such efforts. The important KSAs are suitable for a blueprint of testing methods such as multiple choice. The important tasks are suitable for a blueprint of performance-based testing methodologies.

Needs Analysis Findings
Turning to the survey questions related to certification, respondents were given a list of seven different ways that a system-administrator certification program might affect them personally. The results were generally positive. Only 20% of the sample indicated that certification would either have no benefits or would have a negative effect. In contrast, 72% of the respondents believed that certification would help document their professional capabilities, and 56% believed that it could enhance their job prospects. As you would expect, senior system administrators perceive considerably fewer personal benefits to certification compared to their less experienced counterparts.

Respondents were also asked how a system-administrator certification program might affect the profession. Roughly 22% indicated that certification would have no benefits or could harm the profession. On the other hand, 63% of the total sample indicated that certification would likely increase professionalism and performance standards in the profession. There was a fairly consistent pattern in response to this item for the beginner system administrators to view certification more favorably than more experienced system administrators.

Although the majority of the respondents appeared to see benefits to professional certification, 20—22% is a significant minority of individuals who see no benefits and in some cases negative consequences. In open-ended comments invited at the end of the survey, a relatively large number were quite critical of plans for the development of a certification program.

Despite a fairly strong undercurrent of objections, a clear majority (74%) of the respondents indicated that they would probably or definitely participate in a SAGE certification program. Again, respondents in the beginner level showed the highest propensity to participate (83% probably or definitely participating), and senior-level respondents showed the lowest propensity to participate (66%).

Summary
Analysis of the survey data shows that the system-administration profession is amenable to certification testing. There appears to be a set of tasks and KSAs that are applicable to most system administrators. This assumes, however, that a test can be designed that does not require the examinees to know a specific operating system. The survey data also show that most system administrators support the idea of certification, although a vocal minority do not share this view. The ultimate success of the program would, of course, be determined by several factors such as the quality of the tests, the popularity of the program, its cost, and how well the program is administered.

The 1999 SAGE System Administrator occupational-analysis study has provided a considerable amount of detailed information that will be useful for a variety of purposes. The information can be used for certification, but can also be used as a foundation for other professional development tools and activities. Moreover, additional analyses can be performed to address other specific questions related to the profession.

Because system administration tends to see swift changes in technology, SAGE has been advised it will still be necessary to plan for updating the job analysis on a regular, and relatively frequent, basis. Once every year to two years, an updated automated survey could be administered. With the groundwork done in 1999, future job analysis will be considerably less burdensome. To further reduce the burden, it would be possible to reduce the amount of information collected on each survey, so that ratings on all the tasks and KSAs are not collected every time.

The certification subcommittee cannot complete this work in isolation from its members. Information on this project will continue to be disseminated as it becomes available. Members are encouraged to comment on this effort at <[email protected]> or the <sage-members> mailing list.