The End Users: INLS 92 Spring 2006 Team Project

Assignment 2: Documenting Usage of Chosen Artifact

Due Date: Thursday, February 9, 2006

Artifact Definition

Our group's chosen artifact is a web application called "StudentCentral," (studentcentral.unc.edu) or SC for short, which is used by students of UNC Chapel Hill to perform a variety of administrative and academic tasks. According to UNC Information Technology Services, StudentCentral is a comprehensive student portal designed to provide students with a variety of self-service functions.

"Students can view their grades, class schedules, account billings, any stops and financial aid award information. They can also update personal demographic information such as addresses and privacy flags, apply for student housing, apply for degree candidacy and perform degree audits."

While the functionality of SC is frequently revised improved, as of January 25, 2006, SC contained 37 different functions and links under six broad categories. In addition to SC, there is a separate but related application called "MyUNC" (my.unc.edu), which also serves as a similar portal, but offers a range of customization based on the users' student status, class, major, year and personal preferences. For the purposes of this assignment, we will concentrate on redesigning a truly comprehensive and unified SC, taking both the current SC and MyUNC into consideration. Our aim is one of function--to design an integrated web application where students can perform many, if not all, of the tasks currently offered by both SC and MyUNC.

History of Student Central

Little authoritative history could be located regarding SC. However, before SC and widely-available internet access, students performed many of the functions now offered on SC, in person or via regular (snail) mail. Examples include standing in lines in Woolen Gym to register for class, paying tuition bills and signing up for student housing via the US mail, etc. In October 1990, CAROLINE, a telephonic registration system that allowed students to register for classes using a touch tone phone, debuted. CAROLINE was a revolutionary development and was in many ways a predecessor to SC. Suddenly students were freed from the physical limitations of having to be on campus to register for classes. As time progressed, additional functions were added to CAROLINE beyond simple class registration so that CAROLINE was a telephonic portal to many common tasks. Even while CAROLINE was being extended, UNC was planning the ability to register for classes online. The first online registration system was implemented in the late 90s and co-existed with CAROLINE for a time. CAROLINE was finally terminated on Jan 21, 2004, at which time only 5% of students used the telephone to register. The last hold-out was the University Cashier, which was the last in-person or telephone-only function to be moved to SC. Currently, the hardware supporting the student registration functionality in SC supports 100 simultaneous sessions. Even so, in recent semesters it is not uncommon for 1500-2000 students to register for courses during the first 15 minutes of a registration window. As of January 2006, the University Registrar is anticipating upgrades to the hardware and software comprising SC, but no major changes are planned until 2008 or 2009.

Research Methodology

SC is an application designed for UNC Chapel Hill students, so we limited our observations to either students actively enrolled, or those who may potentially enroll. And because usage of SC is subject to the ebb and flow of the academic calendar (with most usage occurring during registration and grade reporting periods) we compiled our usage profile through a combination of 1) observation, 2) direct interviews of users, and 3) a web survey. These three methods are summarized below with links provided to full raw data yielded from each method.

Student using SC @ House UL Lab 1. Observation: We designed our observation methodology taking the ethnographic methods discussed in class and in our readings into account. Our goal was to glean how the user interacted with the artifact, and the user experience as a whole. The observation component comprised a series of pre-determined tasks we deemed were common tasks most students would perform on SC, and we asked participants to perform these tasks while observing them navigate the artifact. The tasks ranged from simple to complex, and we noted how easily the tasks were performed, any difficulties encountered, and feedback from the participant, either explicit, or in terms of body language, signs of frustration, sighs of exasperation, nervous tics and twitches, etc. The observation periods ranged from ten minutes to thirty minutes, depending on the participants' familiarity with SC and ability to perform the tasks provided. The specific tasks and our full notes are detailed in the observation section.

2. Interview: Our qualitative interview was designed with in-depth and/or open-ended questions about subjects' SC use in terms of utility, style, way of life and social networks. Our interview questions and participant responses are listed in full in our interview section.

3. Survey: Finally, our online survey was designed to measure how participants used and interacted with SC and asked their stated satisfaction rates with various features and functions of SC. The survey questions and results can be found here: Survey Results (external link).

Raw Data

Findings

Sample: Our interview/survey sample consisted of four males and two females, with median and mean ages of 29.5 and 27, respectively. Three subjects were undergrads, two grads, and one subject was a potential student we recruited to gain a fresh perspective and insight. All of our student subjects had been enrolled at UNC since 2001 or later. Our observation sample comprised two females and two males, with median and mean ages of 29.75 and 23, respectively. Two students were undergrads, and two were non- or potential students. Since a majority of UNCCH students are traditional undergraduate students, we hypothesize that our samples were slightly older than the actual student average, but, given that there was little variation in the feedback we received from older and younger students, we feel that our data adequately captures StudentCentral artifact usage among UNCCH students.

Usage: In terms of SC usage, 40% of survey respondents used SC several times per month, 60% rarely or less than once per month, while 100% of survey participants stated they had rarely or never used MyUNC. 60% of survey respondents reported that they used WebMail at least several times per week, while 40% never used webmail, because they had their mail forwarded to another email address. Of all the specific functions offered by SC, no one used any function more than several times per week. Among those both surveyed and interviewed, a majority of our respondents stated that they did not use a majority of the functionality of SC (such as CCI, CWI, housing, financial aid, etc.), while the most frequently used functions of SC, such as registration, grade reports, etc., were used several times a month, and often much more frequently during registration periods and grade reporting periods.

Utility: All three of our research methods (the interviews, observation and survey) clearly indicated that utility was the most important aspect of StudentCentral to its users. Our findings also indicate that StudentCentral is lacking in this regard. All of our study participants indicated frustration with StudentCentral in terms of its performance, navigability and functionality. During our observation phase, all of our study participants expressed clear signs of frustration and dissatisfaction with both general and specific features of SC. According to our research, an overwhelming majority of our subjects were primarily concerned with only three main functions out of the thirty plus functions offered on StudentCentral: Course searches, term registration, and grades. All subjects indicated that these features should be improved, especially in performance, capacity and ease of use. Some of the older students who had used CAROLINE telephone registration liked the ability to register for classes online but still felt that performance ought to be improved. The most common complaint was the slowness of term registration during peak registration periods, and general difficulty in site navigation. Almost all observation and interview subjects complained of errors using the back button of the browser. Several users hinted that SC is really a front-end of some very legacy main-frame apps, hence its primitive look and function. UNC may have led the way when it implemented SC, but it is clearly time to upgrade!

Style: All of our study participants indicated some dissatisfaction with SC's style. Our interviewees described SC as "old-fashioned," "boxy," "a bit rough," and "primitive" with a "Soviet feel." Most of our respondents liked that the page was colored Carolina blue, however, many noted inconsistencies in the look and feel of subsequent pages, with different shades of blue and different page layouts and styles. One participant noted that the logo used on SC is the logo of UNC athletics (the Ram), not the Old Well frequently associated with Carolina academics. Most expressed frustration and dislike for the long list of functions and links, and the "old fashioned," "ugly" table layout. While all participants felt that the aesthetics and style of SC could be improved, all felt that function was more important than looks. Given a choice between having an ugly webpage that performed their registration quickly, or a pretty page that did not perform well, our respondents felt that performance and function were most important, followed by ease of use and navigability. Almost all felt there were too many bells and whistles and that SC needed to be trimmed down, or made leaner. Several survey respondents suggested that SC could be customized and only display functions applicable to the student, depending on student's status. And despite SC's Carolina blue color, most of our respondents did not feel that the color contributed to any sense of belonging or "Carolina studentness."

Networks: We found that students learned about SC through a variety of ways, depending on when and how they entered the university. Several of our traditional student respondents learned of CS while attending CTOPS, or student orientation. However, several other students, one a transfer student, the other, a part-time continuing studies student, only learned of CS accidentally, or through word-of-mouth. These latter students were not given the tutorial that traditional students receive, thus had to learn to use CS on their own, through trial and error. None of our study participants felt that SC was a social tool, although, upon prompting, several admitted that it could have social uses, such as links to "meetups" or help/tutoring with classes. Several students expressed that the only social impact SC had on their lives was a negative one when they were forced to spend hours trying to register for class or lookup grades. The only time SC was brought up in social situations was when respondents complained about SC to their peers or friends.

Way of Life: Most of our respondents did not feel that SC played a significant role in their life as a student, although one student stated that SC's calendar functions allowed him to be better organized. While many felt SC was better than CAROLINE (old phone registration system) for course registration, our respondents seemed to take SC for granted and wanted instantaneous performance and registration. Many felt that the best way that SC could positively influence their lives was by first, reducing the time it took to perform the most important functions in it, and second, by adding more useful functionality and customization, rather than a hodge-podge list of archaic, obscure links to functions no one used. Better course descriptions and search functions, dynamic links to instructor homepages, contacts, office hours, active scheduling, better grade reporting including students' standing in class compared to peers, and customized links and functions depending on students' status and preferences were just some of the suggestions made to improve SC.

Future of SC: Today, many functions offered by SC can only be performed on SC, requiring users to have a computer, web-browser, and access to the internet. We hypothesize that as cellular, wi-fi and personal communication devices become more popular and reliable, many functions will be ported to cellular, blackberry or other wi-fi devices. Another development may be the invention of smart-agents or passive action tools that will allow students to enter their preferences in the tool/medium of their choice, and the agents will perform the actions when network speeds, connectivity, load, class availability, etc. allow. For example, while currently, a student wishing to find out his grade in a class must repeatedly check SC until the grade is posted, in the future we see the student setting a smart-agent that will check for the grade and alert the student through the medium of choice (cell, pda, etc.).

Sources Consulted