10.1. Implementation & Evaluation: how to

10.3. Implementation & Evaluation:... Implementation & Evaluation: Public Communication and Outreach

In the sections below the necessary mechanisms and tools are discussed that should be in place for implementing the Action Plan for ICT as well as considerations that deserve particular attention

Committee for ICT Planning & Development

If present, a “Committee for ICT Planning and Development” would be best placed to oversee and manage the implementation of the Action Plan for ICT. If not present, the first and foremost recommendation of the strategy document may be to install a Committee for ICT Planning and Development with the above mentioned oversight and management mandate and role. Throughout the implementation of the Action Plan the Committee for ICT Planning and Development (hereafter: Committee) should seek to encourage where possible and logical coordination and collaboration between agencies and units of the parliament with regard to: exchanging experiences and best practices; sharing of services and resources in implementing and managing ICT; and better alignment and development of the entire ICT infrastructure as one integral asset ensured by compatibility and interoperability standards.

ICT Governance mechanisms

The Committee may introduce (if not present or fully developed) a number of ICT governance mechanisms or directives for achieving its objectives.

Service Level Agreements: to ensure an adequate delivery of services and level of support from the ICT Department (or Centre or Unit) to the rest of the parliamentary organization, it is recommended that Service Level Agreements (SLAs) are established (ideally mandated and formalized through the intervention of the Committee). SLAs are “agreements” between the service- providing (i.e. ICT Department) and receiving parties (rest of parliament); they specifically indicate what kind of hardware, software, server capacity and technical support should be provided, against what cost and conditions. They also stipulate the parameters of system environments that may be provided as a service including the level of system performance, stability, security and necessary disaster and recovery procedures. Finally SLAs also clearly indicate the type of response that is required from the ICT Department in case of service or system failures, and provide guidance on escalation procedures. SLAs are key in establishing greater trust among parties and encouraging greater cooperation and collaboration in the parliament in terms of the acquisition of ICT services and sharing of resources. SLAs may as such reduce the incentive within departments to create their own independent (and often isolated and disconnected ICT operations)

ICT Procurement Standards and Policies: well considered ICT Procurement Policies and Standards could achieve conformity within the parliamentary (read: departmental) procurement practices in areas such as Intellectual Property (IP) in Software, Liability Coverage; Service Levels from contractors and suppliers (SLAs); Tendering procedures and documentation etc. These procurement policies and standards can be instrumental to make sure that the parliament on the one hands receives the best quality of products and services on the market place and on the other they may prevent an “uncontrolled growth” of the technical infrastructure that is hampered by incompatibility and poor choice of technology standards. Regarding the latter, procurement policies and standards should “tie in” at the technical level with the parliament's enterprise architecture (more under the section below: Consideration for implementing technology) that provides clear inputs for setting technological standards for ICT infrastructural components.

Outsourcing ICT services: In planning and executing ICT projects an issue that will undoubtedly rise is whether to invest in the development of certain in-house skills or instead to outsource these skills. The solution may depend on the cost of investing in these skills versus their market price in combination with the maturity of the country's or region's ICT sector in being able to provide high quality and reliable services. The guideline may be as follows: given that a parliament can contract an external supply of high quality an reliable ICT services at reasonable price, then outsourcing should be considered, albeit in combination with the parliament's in-house ability to manage and oversee the level and quality of the provided services. This implies a certain threshold in the ICT managerial competencies in the parliament i.e. the parliament have the competencies to independently carry out ICT project design and management tasks, and defining user and system requirements. Thus ideally these two areas of competencies are not outsourced. The more technical competencies such as “programming”, “system development”, “installation” and “configuration” may be outsourced to external contractors depending on the market price and cost-effectiveness.

ICT Project management

Concerning oversight and management of projects the Committee for ICT Planning and Development could install (if not present or fully developed) and enforce ICT project management mechanisms and practices.

Project Implementation Board: For managing and coordinating ICT projects an “ICT Project Implementation Board” (PIB) is instrumental. Besides being responsible for monitoring individual ICT projects, the PIB can also re-evaluate priorities, resolve issues and (re)allocate resources across projects. In addition a PIB would also serve to engage and coordinate all stakeholders involved in the implementation of ICT projects; report to the Committee for ICT Planning and Development and escalate project / programme issues affecting the overall progress of ICT projects.

Project management: It is recommended that the parliament invests in developing sound ICT project management competencies through the adoption of project management methodologies that provide tools for planning, cost analysis & budgeting, analysing risks and carrying out project evaluations & lessons learned. The ICT project manager as such should be able to coordinate the formulation of user and system requirements, plan respective work products within the scope of the project and calculate the required time and resources.

Considerations for implementing technology

Concerning the implementation of technology below are three areas highlighted that may be incorporated (if not present or fully developed) during the implementation of ICT projects as they could be instrumental for a well planned growth of the technological infrastructure and delivery of high quality systems.

Enterprise Architecture: Enterprise architecture is a methodology for ensuring that technology directly supports the work of the parliament, where the most relevant data and documents are collected and managed efficiently, and that the specific components of technical infrastructure, such as the PCs, operating systems, and networks are interoperable. Enterprise architecture is especially important when technical support comes from a variety of sources, such as outside donors and the parliament itself. The enterprise architecture ties in with the ICT procurement policies and standards, as it provides technical guidance and rules for adding infrastructure components into the existing architecture. The enterprise architecture would at the highest level be overseen by the Committee on ICT, aided by a dedicated technical working-group that would also take into account considerations for scalability of the infrastructure and the sharing of resources and infrastructure, to achieve a more streamlined architecture. The Committee would not have to deal with the technical details but rather promote the coherence of standards to ensure technical compatibility and interoperability at the parliamentary level (among others through the earlier mentioned procurement standards)

Document standards: Open document standards are important to institutions for helping to control costs (by not being locked into the proprietary standards of specific vendors), creating more capable document management systems, and developing more flexible systems for document and information exchange. However, the use of open standards (such as the EXtensible Markup Language or XML) requires extensive training and knowledge of the most appropriate tools. Because of the additional time that open standards can sometimes require for implementation, it also needs political support. Nevertheless, open standards are important for the future development of ICT in parliaments.

Usability testing: the aim of usability testing is to observe people using systems to discover how they experience using them and find areas of improvement. Usability testing generally involves measuring how well test subjects (i.e. staff that is asked to try out new systems) respond in four areas: efficiency, accuracy, recall, and emotional response. The tests should be carried out during each system implementation and the results of the first test can be treated as a baseline or control measurement; all subsequent tests can then be compared to the baseline to indicate improvement.

• Performance -- How much time, and how many steps, are required for people to complete basic tasks using the system? (For example: search and retrieve a law, create a draft bill)
• Accuracy -- How many mistakes did people make? (And were they fatal or recoverable with the right information?)
• Recall -- How much does the person remember afterwards or after periods of non-use?
• Emotional response -- How does the person feel about the tasks completed? Is the person confident, stressed? Would the user recommend this system to a colleague?

Setting up a usability test involves carefully creating scenarios, or realistic situation, wherein the person performs a list of tasks using the system being tested while observers watch and take notes. Several other test instruments such as scripted instructions, paper prototypes, and pre- and post-test questionnaires are also used to gather feedback on the product being tested.

10.1. Implementation & Evaluation: how to

10.3. Implementation & Evaluation:... Implementation & Evaluation: Public Communication and Outreach