• Determine the UCD campus needs with regard to instructional technology.

• Conduct a situation inventory on and off campus to identify technologies that are in use, or might be useful for teaching and learning and evaluate these technologies.

• Through consultation with faculty, staff and students, through pilot testing and prototyping, and through work with national efforts such as the National Learning Infrastructure Initiative and the Instructional Management System, create a comprehensive definition of the tools, architecture and infrastructure for a learning environment that will match the needs and unique culture of the University of California, Davis campus.

Needs Assessment

Manage complexity: if possible, hide complexity without reducing functionality; if not, minimize it with the right functionality trade-offs.

• Help us do more re-usable work
• Make familiar tools useful for a wide range of tasks, to reduce the need to learn new technologies.
• Select/develop tools which allow the re-use of work (through import/export utilities, or standard protocols which maximize compatibility between tools).
• For common tasks, provide a range of tools to choose from, from introductory (with limited features, but high ease of use) to advanced (more complicated to use, but more feature-rich).
• Make the administrivia trivial to do.
• Un-bury us­make it easier to manage the information flow (including e-mail).
• Optimize flexibility.

Help us manage dissemination of and access to information.

• Help faculty and students easily publish and distribute class-related materials.
• Make it easier and cheaper to distribute, collect and manage class materials.
• Help us control access to our intellectual property, allowing us to share access with a defined community of scholars.

Help us communicate and collaborate.

• Help connect faculty to their students.
• Help connect students to each other.
• Help us share work that is re-usable.
• Create an environment which provides communication and participation options for more reticent students.

Design principles and institutional view Because the following needs are ubiquitous or based in legal/policy requirements, all services and systems should be responsive to them:

• Support the ability to work from anywhere, at anytime, at our convenience.
• Provide for secure and verified identity to document the source of messages and approval actions.
• Protect privacy.
• Protect intellectual property rights.
• Support all platforms.
• Comply with ADA requirements.

Responsive Technologies

Portals Course Management Software Databases Web Server/Middleware/Database Integration Specialized databases and repositories (e.g., Image databases)

Authentication, Authorization & Security Faculty-Friendly Middleware Meta-data and search functions Common file space Mail clients (with management utilities and middleware) Commercial presentation software (with utility middleware)

Table 1 lists and describes in more detail the needs identified above, provides the information source (how was the need identified), lists classes of technology that the LEAD team believes could be responsive to the needs, as well as the alternatives available in each technology class.

With the campus needs and potentially responsive technologies identified, the next steps are to evaluate which technologies and services should be pursued, and how much in the way of resources should be invested in them.

The range of options appears to be:

• Do nothing (the technology isn't responsive to any need).
• Research and monitor the technology (if it is an experimental technology).
• Prototype the technology (if it is an emerging technology).
• Pilot the technology (if the need is great enough, and it seems like the technology is mature enough to begin deployment).
• Bring the technology into limited production (once the bugs have been worked out through pilots, uncover the implementation "bugs").
• Bring the technology into full production.

On what basis should selections be made among these options for each of the responsive technologies given above? After that choice is made, who implements the option, and on what scale (department-wide, division-wide, campus-wide, system-wide)?

Decision-Making Rubric

• Criticality of Needs: How critical is the campus need which the technology meets?
• Technology Readiness: What is the relative "readiness" of the technology to meet the need?
• What is the willingness to adopt the use of the technology?

Criticality of Needs

• affecting the campus' ability to carry out its central missions (e.g., over-enrollment of large general education classes and constraints on classroom facilities), and evidence that the need/problem will persist and worsen (Tidal Wave II);

• shared by all the relevant populations­faculty, staff, students (e.g., common behavior and needs as mobile workers);

• important enough that several departments are already individually investing time and resources in meeting it, or asking for campus resources to address the problem (e.g., multiple requests for IUC funding for image databases);

• arising out of increasing expectations that are formed in the mass market (e.g., student expectations about "e-commerce" capabilities that the campus should support, based on what is commonly possible in the market­ability to pay registration or housing fees);

• associated with a strategic campus initiative;

• confirmed by the willingness of a specific campus unit or division to sponsor a project to address the need, through the use of technology.

The potential ranking of the need is along a continuum, from "nice to have," to "have to have to carry out essential campus activities." In addition, needs have a vector ­ are they increasing, stable, or decreasing?

The experts in needs are the users themselves, and the bodies which represent them, including the Academic Computing Coordinating Council and the Administrative Computing Coordinating Council.

The LEAD team sorted the needs in descending order by priority or "criticality" based on AC4 review, the results of the surveys and interviews, the results of the LEAD Think Tank, and the responses of a final focus group composed of faculty, staff, and students, as follows:

• Manage complexity. This was identified as the most critical need. Criticality is in the upper quadrant (very high) and increasing.

• Help us manage dissemination of and access to information. While not as critical as managing complexity, this need was moderately high, and also increasing.

• Help us communicate and collaborate. This need, while increasing, is still in the "nice to have" quadrant.

• Design principles and institutional view. This is a special class in needs. Individual faculty, students and staff are not likely to identify this type of need (i.e., conformance with ADA requirements, protecting intellectual property rights) as critical. However, because of legal or regulatory requirements, the institution itself often has a responsibility to be responsive to these needs.

Technology Readiness

1. At least one commercial product exists.
2. Stable, accepted market standards exist.
3. The technology is accepted and used widely at comparable institutions of higher education (adoption rate is high).
4. A single, strong offering dominates the market, or multiple offerings are available in the market.
5. Implementation feasibility is high.
6. Technology is scaleable, replicable, extensible.
7. The technology can leverage existing infrastructure and investment.
8. The technology can be leveraged to support other functions.

The four major classifications for responsive technologies and the characteristics of each are:

Experimental -- No commercial products exist (or are in beta form only); market standards aren't even under development ; the technology is not in wide use; implementation feasibility is low because of complexity, bugs, or poor user interface; or the technology has not been demonstrated to be scaleable, replicable, or extensible.

Emerging -- A number of early commercial products (with limited features) may be offered commercially; market standards are under development but still volatile; the technology is not in wide use by comparable institutions; implementation feasibility continues to be low; or the technology has not been demonstrated to be scaleable, replicable, or extensible.

Maturing -- There may be several commercial products available; market standards are becoming more stable, although there may be two major standards choices competing in the market; implementation is more feasible because software bugs have been eliminated in the commercial products; user interfaces are more acceptable and more features are supported; a number of early adopter campuses have implemented use of the technology; the technology appears to be scaleable, replicable and extensible.

Mature -- There are a number of commercial offerings; standards are stable and have been formally accepted by standards body (IEEE); the technology is accepted and used widely at comparable institutions of higher education; implementation feasibility is high; technology is scaleable, replicable, and extensible; technology can leverage existing infrastructure and investment.

Willingness to adopt

Innovators -- 7.5% of the population is willing to innovate in their use of technology, even to the extent of developing it (since the market typically doesn't have any offerings at this point in the development cycle). This population has a high tolerance for risk, and for a steep learning/implementation curve, and may be interested in technology in general.

Early adopters -- 13.5% of the population is willing to adopt a relatively new technology in advance of their colleagues. Again, because the technology is somewhat immature, members of this population experience a steep learning/implementation curve.

Early majority -- 34% of the population is willing to ride along with the first large wave of adopters. These faculty have less tolerance for risk, are more expedient (interested in what technology can do for them), are generally not interested in technology for its own sake, and require that the technology be easy to use, and supported.

Late majority -- 34% will take advantage of the experiences of previous adopters, and maturation of technology by waiting until a majority of the membership has implemented use of the technology. This population is risk-averse, and completely expedient with regard to technology (what can it do for me, right now, with a minimum investment in time and energy?)

Late adopters -- 16% of the population will resist any adoption of technology until it is imposed on them by policy or peer pressure. This population may be uncomfortable with technology, or have pressing priorities that technology may not further. A current example of this population are those faculty who aren't yet using electronic mail for instructional purposes.

Process Used By the LEAD Team

• Evaluated criticality of needs.
• Identified technologies, and confirmed that the technologies were responsive to the need(s), and that all appropriate technologies and alternatives had been identified.
• Evaluated the "readiness" of the technology.
• Mapped the needs to the responsive technologies to develop the suggested implementing action (research and monitor, prototype, pilot, limited production, or full production).

Mapping Needs to Responsive Technologies

An example of the mapping process is given below:

1) Create a vertical axis, and rank the needs along it, from low criticality to high.

For example, the following vertical axis goes from low need (e.g., support distance learning, which is probably not a high priority for our Research I, residentally-based campus) to high need (we need to do more with the same classroom resources by providing some way to respond to over-enrolled general education classes and Tidal Wave II). Helping faculty communicate with their students is evaluated as an important need.

• Robustness of the technology (scalability, extensibility, interoperability, full feature set).
• Maturity of the technology (stable standards, multiple commercial offerings, adoption by comparable universities).
• Implementation feasibility of the technology (cost, whether it fits the UCD environment and culture, if we have sufficient infrastructure, support).

For example, we could look at electronic mail as a technology that is responsive to the need for faculty to communicate with students conveniently and effectively. In evaluating electronic mail along three horizontal axes representing the criteria above, we would likely agree that it isn't perfectly robust (it lacks some of the features, such as ability to have graphics in our messages), but it is pretty mature and we've already demonstrated that the implementation feasibility on this campus is high.

If we put the two axes together, the intersection of the need and the technology would be in the upper right quadrant of the graph (see point "D" in the diagram below):

The other dimension is trend over time, which is represented by a trend line. An upward trend line represents an increasingly critical need, and a rightward direction on the horizontal axis, a technology that evolving and maturing.

An additional general recommendation was also developed:

1. Research and monitor experimental technologies, where the matching need is not critical.
   Example-Virtual Reality as a tool for simulating complex chemical interactions.

2. Prototype emerging technologies, where the matching need is increasing, but not yet critical.
   Example-Image Databases as a tool for sharing resources.

3. Pilot emerging technologies, if the matching need is both critical and increasing.
   Example-Common file space for electronic distribution and collection of class materials.

4. Go into limited production once the technology is determined to be maturing, if the need is critical.
   Example-Use portals and middleware to manage complexity.

5. If the need is not yet critical, but is increasing, pilot the maturing technology.
   Example-Pilot PKI for secure e-mail.

6. Support full production only where the need is critical and the technology is mature.(E-mail)

7. Even if the technology is mature, if the need is declining, begin execution of the exit strategy.

Recommendations - Learning Environment Architecture

1. In order to help manage complexity for faculty, develop a class management portal system. Because the need to manage complexity is critical, and portals are a maturing technology, put this system into limited production as soon as possible. The faculty class management portal system should automate, organize and present all class management functions using integrated web pages and middleware.
   1. The full set of functions for class management would include:
      • Defining/reviewing course goals and objectives.
      • Selecting general content materials.
      • Identifying textbook and making arrangements with bookstore.
      • Finding web sites and other on-line resources.
      • Developing course structure.
      • Developing and distributing teaching plan (syllabus).
      • Planning detailed lectures.
      • Developing and distributing lecture notes, class handouts and other course materials.
      • Planning TA activities (define roles, interaction plan).
      • Designing student interaction and collaboration, and selecting communication tools; setting up mechanisms for communication with students (e-mail lists, chat, newsgroups, whiteboards, etc.)
      • Designing assignments.
      • Designing labs.
      • Designing tests.
      • Creating web page for course.
      • Determining access rules for on-line materials.
      • Designing site.
      • Authoring static pages.
      • Setting up dynamic structures for web page generation (databases, forms, etc.)
      • Preparing graphics and including them on the web page.
      • Transferring files to web server; activating site.
      • Identifying desired classroom on basis of needed characteristics, and generating reservation request.
      • Reserving computer labs.
      • Managing web site.
      • Managing class list (adds/drops); monitor for prerequisites.
      • Managing student interactions and collaborations.
      • Distributing assignments.
      • Collecting assignments.
      • Tracking assignments, and maintain record of evaluation (grades).
      • Administering tests.
      • Grading tests.
      • Recording and tracking grades.
      • Reporting final grades.
      • Archiving class materials.
      • Cloning new class from existing class.
      • Evaluating web site (effectiveness of material, use of material, etc.).

   2. This system should be developed incrementally, with existing critical functions organized in this manner, and additional functions developed and set up for access through the portal on a prioritized basis. The current MyUCDavis could be the basis for this portal system. Include the Teaching Resources Center in the prioritization process by adding TRC representation to the Steering Committee or Advisory Committee.

   3. Leverage the use of existing databases (e.g., Student Information System) through the use of middleware, standard formats, and interoperable systems-without constraining faculty ability to manage classes. For example, class rosters should be generated automatically from the Student Information System, but faculty should be able to add and delete students from their class list as needed.

   4. Even if existing (and new services) weren't developed in an integrated manner, use middleware and the portal system to integrate them from the faculty person's perspective.

   5. Roll out new services for faculty using the portal system.

   6. Work with departmental technical support coordinators to provide access to locally-developed class management services through a defined area of the same portal. Evaluate these for campuswide use.

   7. Provide flexibility while managing complexity by permitting faculty to choose the following options for their course web pages:
      • Minimal web page, generated from existing database.
      • Limited customization, with data from existing databases, and additional data collected through simple web forms that faculty or departmental staff complete.
      • More complex course web pages, generated by commercial course management system (see recommendation #2 for more details).
      • Completely customized course web pages, whether generated through static html pages, or dynamically generated with a database, middleware and web server.

2. Conduct feasibility pilots on two Course Management Systems as recommended by the Integrated Course Management team (a task group of the LEAD team), and evaluate the extent to which they could help faculty manage complexity.
   1. Set up implementation feasibility evaluation criteria, consulting with faculty, departmental technical staff, I.T. Client Services staff, other I.T. staff, and students.

   2. Pilot the use of CourseInfo in 2 to 4 classes, and evaluate according to criteria.

   3. Pilot the use of WebCT in 2 to 4 classes, and evaluate according to criteria.

   4. On the basis of the pilots, report on the following issues and make recommendations to the decision-making bodies (Technology Infrastructure Forum, AdC3, AC4) on the following matters:
      • Should UCDavis implement a course management system (technical feasibility and implementation feasibility)?
      • What support and infrastructure would be necessary to use a course management system (and what would that cost)?
      • If so, how should such a system be implemented (centrally, distributed in departments, hybrid)?
      • If so, should the system be one of the two commercial packages, or a component-based system developed locally?
      • If a commercial product should be used, which course management system would provide the most appropriate support for UCDavis courses, and which is preferred by UC Davis faculty, students and technical staff?

   5. On the basis of the pilots, and additional consultation with departmental technical staff, develop a Recommended Solutions document for departmental and campus course web page generation, maintenance, and archiving.

   6. On the basis of the pilots, additional consultation with departmental technical staff and usability experts, develop CourseWeb Page Design and Maintenance Guidelines.

3. In order to manage complexity for students, develop a similar student portal system (which is fully integrated with the faculty class management portal system), providing the following functionality:
   1. Access course web pages in one place, for all classes in which the student is registered, whether the pages are generated from the student information system, a course management system, or a customized system used by their faculty.
   2. Access e-mail through web interface.
   3. Provide support for publication of student course page (notes, individual projects, etc.)
   4. Provide support for project web pages for student collaborations.
   5. Provide controlled access to student web pages.
   6. Provide for the archiving of student web pages in their own directories (so that if the class archive is deleted, if the student wants a copy of their web work, they can still maintain one.)

4. In order to manage complexity for technical staff, develop a similar technical staff portal system through the TSP consultation process.

5. In order to help faculty and students manage the dissemination of and access to information, evaluate different methods of providing common file spaces for classes (areas that a faculty person can set up for posting assignments and handouts for downloading by students; and that students can access for turning in assignments or conducting collaborative group projects)
   1. The desired characteristics for this file space are:
      • Ease of use: Simple user interface (GUI), based on existing conceptual models (e.g., copying files by drag/dropping into folders)
      • Leverage existing infrastructure: Automatic set-up for each individual, and each class based on existing databases..
      • Easy forms-based set-up for groups (informal study groups, as well as more formal faculty-organized groups).
      • Robust access control with a simple user interface.
      • Student privacy policies supported.
      • Access to files without downloading.
      • Extensibility, scalability, and sustainability.

   2. Conduct feasibility pilot of AFS and other systems to evaluate technology readiness, and performance according to the evaluation criteria.

   3. Include common file space functions in the evaluation of the course management systems. (See recommendation #2 above).

   4. In order to help students disseminate information about themselves, and promote informal collaborations (personal web page, personal portfolio, collaborative web pages, etc.), conduct a pilot project to evaluate services that provide web space to individuals. (e.g., I-Drive). In addition, evaluate the service to determine the extent to which it can support common file space functions.

6. Provide authentication and authorization system sufficient to control access to intellectual property, and protect the campus' legal interests with regard to copyright protection and limited access to copyrighted documents.
   1. Pilot the use of personal digital certificates (using Public Key Infrastructure).
   2. Research and monitor the development of other encryption, authentication and authorization systems.
   3. Continue to develop and extend the existing distributed authentication service.

7. Support mobile work for faculty, students, and staff.
   1. Provide campus-wide production service for DHCP.
   2. Include remote access guidelines in the Course Web Page Design and Maintenance Guidelines.
   3. Include remote access support in authentication and authorization systems.

8. Evaluate web server/middleware/database combinations and develop selection and development guidelines as part of the Course Web Page Design and Maintenance Guidelines.
9. Provide design guidelines for campus developers of instructional systems. In addition to the Course Web Page Design and Maintenance Guidelines,
   1. Develop Instructional Systems Interoperability Recommended Solutions document for technical staff, incorporating existing IMS specifications.
   2. Monitor development of IMS specifications, and relay appropriate information to campus developers.

10. Provide collaboration and communication tools for informal collaboration.
    1. Working with the Teaching Resources Center, T.A., and discussion groups, prototype informal collaboration tools.
    2. Evaluate the technical feasibility of their use.
    3. Make recommendations as to pilots, as appropriate.

11. Evaluate digital repositories for instructional use.
    1. Conduct prototyping project of digital image repository for instructional use.
    2. Pilot should involve library personnel with regard to digital library issues, including archival and data retrieval.

Institutional Infrastructure & Policy Issues

Faculty Support

1. non-existent.
2. If they exist, not visible.
3. If they exist, not integrated with each other.
4. If they exist, aren't local enough for easy access.

Currently, faculty members find their primary source of assistance in their departmental or college technical support staff. This is true for all areas of support, including the development of web pages and other support for hardware, software, courseware, and networks. Interviews confirmed that for most faculty, this both the actual and preferred state of affairs. On the other hand, the technical support staff survey and interviews indicated that support of instructional technology for faculty was not clearly identified as the departmental technical staff's responsibility, and if they were supporting faculty for academic activities as well as administrative staff, they were beginning to feel overwhelmed. Extending the current cooperative efforts of the TSP and the TRC could mitigate this situation, as could the departmental technical staff portal system.

In addition, there is no clear mechanism for ongoing assessment of faculty needs, or for integration of new applications (with each other, and with existing applications) developed to meet those needs. In addition to ensuring the appointment of TRC representatives on committees such as the MyUCDavis Steering Committee, the LEAD team recommends the formation of a group made up of representatives from all campus units which provide instructional support to faculty (including TSC's), for the purpose of identifying needs and communicating these to various project steering committees and service units involved.

Policy Issues

In addition, faculty and staff complain that ownership rights are not clear with regard to software, courseware, or computer-based instructional content which has been developed with campus resources. The undefined nature of this area is a barrier to development of this type of content. This barrier would be eliminated or reduced through the development of clear, formal institutional policies concerning intellectual property rights as they relate to the creation of computer-based instructional tools and resources.

Finally, during the LEAD consultation process Library staff indicated a strong concern that projects were building extensive databases or repositories of information without considering data archiving and retrieval for future use. The conversion of many critical academic resources, such as slides or course materials to digital format is being done without addressing the long-term accessibility or viability of that resource. The LEAD team recommends that the Library take the lead in identifying and using national best practices and emerging UC California Digital Library and national standards (such as XML and metadata strategies), and in developing campus policies and design guidelines for database owners and creators that will preserve data integrity and permit access to the data in future years regardless of the technology used.