Following enhancements to the basic pilot service resulting from the initial user feedback an enhanced pilot service will be tested again during March-April 2007.
Full results are available from the links above. Brief summary results are provided below.
1. Web Based Questionnaire
50 users participated in the web based survey. The majority of users were based at UK HE institutions (58%). Information Professionals made up the bulk of respondents to the survey (46%), with researchers (18%), lecturers (10%) and students (12%) being less well represented.
- 94% of respondents believed there was a need for an engineering cross search service.
- 79% of respondents indicated that they were able to find useful information using the basic pilot service.
- Respondents rated the content searched by the basic PerX pilot favourably (OK 33%, Good 52%, Very Good 6%).
- 89% of respondents agreed that adding more content to the basic pilot would increase its usefulness (Agree 37%, Strongly Agree 52%).
- The basic pilot service was generally found to be easy to use (83% agreed or strongly agreed).
The main suggestions for improvements to the basic pilot service were;
- Improved Content - more specialised engineering sources are required.
- Search Default - should be 'All'.
- Advanced Search Options - improve advanced screen layout/options.
- Results Display -
- Provide combined results set rather than results from individual targets.
- Offer clustering of results via resource type.
- Improve ranking of combined results sets.
- Provide full text indicators.
- Alerting - Offer some form of notifications service.
2. Focus Groups
A focus group consisting of 6 engineering lecturers and researchers was conducted at HW University. The principle aims of the focus group were to ascertain;
a). Participants initial feelings about the pilot.
b). Participants feedback on pilot content.
c). Participants views on a subject based approach.
d). Participants thoughts on a range of developmental possibilities.
a). Participants initial feelings about the pilot
- Fast.
- Coverage is patchy.
- The pilot is more focused than some general search engines.
- Default should be search 'All'.
- Quite a clean and intuitive user interface.
- Offer a spell checking facility.
- Support Google type syntax e.g. *, +, -, ?
- Remove collections with zero hits from results page.
- Facilities are required for saving records, conducting fielded searches, refining and sorting results.
- Full Text availability indicators would be useful.
- Post search clustering may be more valuable than pre-search clustering via resource type.
- Language Translation for Technical Terms.
b). Participants feedback on pilot content
- Current Coverage is insufficient.
- Full text indexing is desirable where possible.
- Ok to include searching of collections where full text is not available.
- More quality subject specific content is required. The following sources were explicitly mentioned;
- IEEE Xplore
- IEE
- RS Datasheets and commercial applications notes
- EPSRC Grants on the Web
- EPSRC Final Reports
- Wikipedia
- Flickr
- Google/Google Scholar
- Current engineering projects
- Potential Funding sources
c). Participants views on a subject based approach
- Google Results are already very good - competition is high.
- Important to stress that Perx is searching content sources which may be difficult/impossible to find via Google.
- Work is required to increase the relevance of Perx hits.
- Possible advantages of the Subject Based Approach include; efficiency/time savings, searching with subject specific terms, higher quality hits, novel content which is hard to find using search engines.
d) Participants thoughts on a range of developmental possibilities
Participants were shown ten developmental possibilities and were asked to rate them.
The Top 5 Developmental Possibilities (score out of 30) were as follows;
- Additional Pilot Content (29)
- Full Text Indicators (29)
- Refine Search (26)
- Record Selection (25)
- Merging Sets (24)
Pilot enhancements are currently under development. Testing and feedback on the enhanced pilot is due during March-April 2007.