It is very important for facilities to have some measure of performance of the effectiveness of the service. Although this can be ephemeral, there are metrics that may be derived that will help the manager of any core facility justify its existence. Amongst the factors that can be considered are:
- Number of users (analysers vs sorters)
- Level of usage (regular vs occasional)
- Number of Laboratories and types of research
- User Publications
- User Theses
- Hours spent training
- Consultation (Internal and external)
The more numbers that can be derived the better as this will enable the impact of the facility to be gauged.
All core facilities will in some way monitor the amount of usage of all equipment in the facility. This is useful in several ways – it can show how well used the equipment is, it can be used to monitor trends in usage (if a particular piece of equipment shows an increased demand, this can be used in a justification for additional hardware), it can be used to monitor the types of experiments that the equipment is used for (is there, for example, an increase in multicolour phenotyping where the current equipment may soon be limiting?). Usage of cell sorters can also be looked at it the same way. There are several ways that usage can be monitored either in the form of a paper diary, an online web-based booking system or a log-in system for each cytometer.
Case study: Derek Davies, London Research Institute, Cancer Research UK
“In my laboratory, we use an on-line booking system (Resource Scheduler). At the end of each month I can automatically generate the hourly usage of each individual or each group and determine also what type of usage that is. when I collate metrics of machine usage I apply different weights depending on whether the user was self-operating, whether the user was in training or needed staff help or where samples were run for the user by Laboratory staff. In terms of cell sorting, I also have to factor in machine preparation and decontamination time. Having these figures to hand makes it easier to decide when we are approaching a critical phase i.e. when lead time for bookings extends to unacceptable levels and we can do something about this before the crisis hits!”
People Cube (Meeting Maker/Resource Scheduler)
Once usage has been established, it is easier for the facility manager to determine levels of re-charge for the Laboratory equipment. Balancing the budget is clearly important although it is good to establish with a financial director what would happen if there is either a shortfall or a surplus in a particular year. It is also prudent to establish whether income/expenditure will be judged over a single financial year or over another fixed period, say 3 or 5 years – this is particularly important in the early days after the establishment of a facility.
Although physical time spent on cytometers can be quantified relatively simply, there are other aspects to the core facility that cannot but these should equally be taken into account and include:
- Independent work (primary research, technique optimisation, practical troubleshooting etc)
- Administration (this will vary with level of staff but for the core manager can take some time!)
- Attendance at seminars (internal)
- Attendance at meetings
- Internal Laboratory meetings
- Sample preparation
- Data analysis with users
- Data presentation (Powerpoint, DTP etc)
Even an established facility cannot afford to sit on its laurels, it must constantly try to make improvements. One way of approaching this is to periodically survey the userbase to ask whether the service is as expected, what could be improved and are there any areas that aren’t covered. This can be a fairly simple questionnaire mailed to users, PIs and others. In general, those facilities that have tried this approach don’t report a very large response which in some ways can be gratifying. Basic housekeeping questions that may be addressed include:
- How long have you been using the facility?
- What is your average weekly/monthly usage?
- Which equipment do you use?
- Which techniques do you use?
Questions can be designed to get a feel for the standing of the facility:
- How do you rate the facility (eg scale of 1-5)?
- Do you consider the recharge rate to be poor/fair/good?
- How do you consider the knowledge of the facility staff?
Other questions may be left open-ended and require a written response:
- Which new techniques do you anticipate using in the coming months?
- What areas of the facility could be improved?
- What would you like to see introduced?
A lot of information may be gleaned from such surveys although there will always be users who find fault and always be users who can see no wrong in the facility so sometimes excluding these outliers from a summary of the responses gives a truer picture!