Appendix 1
Report
into the Surveys of EDS Service Users and IW Foster Carers, June – September
2003.
Between
June and September 2003 a telephone satisfaction survey of a sample of
Emergency Duty Service (EDS) customers was undertaken. At the same time a postal satisfaction
survey regarding ‘out of hours’ support received from both the EDS and from the
Fostering and Adoption Services Team (FAST) was sent to all
Isle of
Wight Approved Foster Carers, including Family Link Carers.
The purpose
of this report is to explain the methodology of both surveys, to bring together
their results and identify areas for improvement, especially in relation to the
support of Foster Carers outside office hours.
Rationale. The underlying principle of the EDS survey was to test
service users responses against relevant SSI Inspection Standards for Emergency
/ Out of Hours arrangements:
1.1 Public information is provided on the emergency
out of hours service, which includes access arrangements and any eligibility
criteria that apply.
1.2. Service users can access the emergency out of
hours service quickly.
1.3 Service users receive an appropriate response
which is effective in reducing risk and stabilising an emergency.
1.6 Service users express satisfaction with the
service provided and are positive about the outcome of their enquiry.
5.7 Service users say that they were treated with
respect and that services provided were appropriate to their particular needs.
7.5 Information from complaints, both from service
users and other agencies, is used to review the effectiveness of the service,
and indicate any changes necessary.
A largely self selected sample of service users were telephoned the next working day and asked a series of questions about a their experience of contacting and receiving a service from the EDS.
Process. During the usual operation of the EDS, after responding to social care emergencies outside office hours, the Filtering Officer (Duty Manager) would ask the service user if they would be willing to be contacted by telephone for the purposes of the survey.
Details of the persons willing to be contacted were passed
on, the next working day, to a member of staff from the Social Services ICT
Section who is independent of the EDS.
She then telephoned the individuals and recorded their responses to the
survey questions and scaled their satisfaction with each element of this
experience. These questions are
detailed below under ‘survey results’.
Target Group and Sample Group. The total number of referrals made to the EDS during the
period of the survey was 420. As is
always the case, the rate of re-referral of the same person in any one month
ranged between 18% and 32% (ave 25%), so these were excluded in order to avoid
double counting.
An even larger proportion of the total referrals (32%) had
to be excluded because they were agency referrers, e.g. police officers or
nurses, who were not going to be on shift to enable them to be contacted. Separate consultations with these agencies
will be required to ascertain satisfaction with the EDS. The remainder, a total of 205 service users,
therefore constituted the target group.
Within this group, a proportion were either not wishing to
be contacted with the questionnaire or were deemed by the Filtering Officer as
likely to be distressed or confused by a telephone call. The total number of respondents that
completed the survey was 45; this represented a sample of 22% of the target group.
Telephone Survey Results.
|
Users that found contacting the service VERY EASY |
30 |
|
Users that found contacting the service EASY |
15 |
|
Users that found EDO telephone number from CHILDRENS SERVICES |
10 |
|
Users that found EDO telephone number from PREVIOUS USE |
1 |
|
Users that found EDO telephone number from WIGHTCARE |
25 |
|
Users that found EDO telephone number from OTHER |
1 |
|
Users satisfied with promptness of response from Operator |
45 |
|
Users satisfied with promptness / response from FO |
45 |
|
Users satisfied with promptness / response from member of staff who
visited (16/45 visited) |
16 |
|
Users not visited but satisfied with response |
29 |
|
Users who said staff completely understood the emergency being
reported |
45 |
|
Users that said referral appropriately dealt with by EDS |
45 |
|
Users who would definitely recommend this service to someone else |
40 |
|
Users who would consider recommending this service to someone else |
2 |
Percentage Satisfaction Scores and Comparators. Below is a table that gives the
above results a percentage of those completing the survey, column EDS. It also gives, for comparison, the
satisfaction rating given in response to similar questions in the two surveys
mentioned above.
In 1999 the DoH commissioned research into a sample of eight
Local Authorities emergency arrangements the results of which were published
under the title Open All Hours.
Satisfaction percentiles from this study are in the column marked OAH.
A similar, but smaller study of 18 respondents, was
undertaken as part of the Best Value Review of the IW ‘Out of Hours Service’,
published in March 2001. These results
appear in the column marked BVR.
|
Subject of question
|
EDS |
BVR |
OAH |
|
Users that found contacting the service
VERY EASY |
100% |
100% |
79% |
|
Users that found contacting the service
EASY |
|||
|
Users that found EDO telephone number
from CHILDRENS SERVICES |
22% |
63% |
59% |
|
Users that found EDO telephone number
from PREVIOUS USE |
2.2% |
||
|
Users that found EDO telephone number
from WIGHTCARE |
55.5% |
||
|
Users that found EDO telephone number
from OTHER |
2.2% |
28% |
0% |
|
Users satisfied with promptness of
response from Operator |
100% |
83% |
48% |
|
Users satisfied with promptness /
response from FO |
100% |
95% |
70% |
|
Users satisfied with promptness /
response from member of staff who visited (16/45 visited) |
100% |
100% |
N/A |
|
Users not visited but satisfied with
response (29/45 not visited) |
100% |
94% |
44% |
|
Users who said staff completely
understood the emergency being reported |
100% |
|
|
|
Users that said referral appropriately
dealt with by EDS |
100% |
83% |
48% |
|
Users who would definitely recommend
this service to someone else |
88.8% |
|
|
|
Users who would consider recommending
this service to someone else |
4.4% |
|
|
Analysis. How do these results compare with
the outcomes required within the SSI Inspection Criteria?
1.1, ‘Public Information’ etc. This survey shows that the largest proportion of service users
have learned how to contact the EDS though previous contact with Social
Services. This is in line with both
previous studies. Given that 95% of service users are existing Social Services
clients this is not surprising; but it may also indicate that improvements can
be made in the dissemination of information on the EDS within other agencies,
e.g. GP Surgeries.
1.2, ‘Quick Access to the Service’. 100% of service users said that they had
found it easy or very easy to contact the Service and 100% were very satisfied
with the promptness of response from the Operator. Both these findings compare well with previous studies and
demonstrate that current arrangements for contacting the EDS are
satisfactory. These standards are
subject to constant monitoring.
1.6, ‘Service Users Express Satisfaction with Service and
Outcomes’ and
1.3, ‘To Reduce Risk and Stabilise the Emergency’. 100% of service users stated that they were
satisfied with the response of the EDS both in the case of the FO dealing with
the query or if a member of the EDS visited.
These responses also compare very favourably with the previous studies
and indicate that expected response times and thoroughness of the intervention
are being well maintained with successful outcomes implemented. Supervision of these elements is ongoing.
5.7, ‘Service Users Treated with Respect and Services
Appropriate to Needs’. The specific
question regarding being treated with respect was not asked but the survey
results and the comments made, see below, indicate that service users were
satisfied in this regard. 100% of
service users stated that their referral
was appropriately dealt with by EDS, i.e. ‘appropriate to their needs’.
7.5, ‘Learning from Complaints’. Although service users were given the opportunity to complain during this survey none were made and all comments were complimentary. This is not to say that there are never complaints about the EDS; 3 have been dealt with in the past 18 months all of which have been satisfactorily resolved. Review of relevant areas of the service has followed with some changes made.
Comments.
As stated above, all respondents to the survey were given the
opportunity to make comments about the service, be they praise or
complaints. Not everyone wished to take
this opportunity and some comments were repeated and so only appear once:
“They are wonderful. Can't express enough how efficient the service was from beginning to end. Very good service. Thank them for extra care. Felt safer with sitter. Please thank them. Always helpful.”
Conclusion. Whilst these satisfaction scores and comments are complimentary, they do not provide grounds for complacency. These aspects of the Emergency Duty Service that have been examined in the survey, along with all of the elements that comprises it, will be subject to ongoing monitoring and continuous improvement through implementation of the Best Value Improvement Plan (presented to the Select Committee in April 2003).
Emergency Duty Service and Fostering
and Adoption Services Survey of Foster Carers.
Methodology. It was decided that, in order to reach all Foster Carers during the consultation period, this survey would consist of a postal questionnaire that had been jointly designed with the Fostering and Adoption Services Team (FAST). The aim of this questionnaire was to ascertain the level of usage and satisfaction with both the Emergency Duty Service (EDS) and the FAST Support Workers, who are available outside office hours via a mobile telephone.
Process. This five-point questionnaire was
posted to all IW Approved Foster Carers and Family Link Carers, a stamped
addressed envelope was included. All
replies were anonymous and were returned to the Manager, EDS, who collated the
responses and produced this report.
Target Group and Sample Group. The target group
was all IW Approved Foster Carers and Family Link Carers, a total 153.
Telephone Survey Results. Disappointingly, only 21 (13.7%) of the surveys were returned and of these 11 Foster Carers said that they had contacted the Emergency Duty Service and 12 said that they had contacted FAST for advice and support out of hours.
Not all Foster Carers chose to score their satisfaction with either service, the satisfaction ratings are therefore based upon 9 Foster Carers (5.8% of the total) scoring satisfaction with FAST Support and (a different) 9 scoring satisfaction with the EDS. In both cases results are influenced by the nature of the survey in that only the views of those Foster Carers who were motivated to return the scored forms were represented.
A more pro-active random selection would have produced a larger sample and a wider range of responses which, when averaged out, could have produced a more useful result.
A wide range of comments were included (see below) and these would appear to be a good starting point for service improvements.
Percentage Satisfaction Scores and Comparators. Comparators have been difficult to find for EDS support, but there has been a recent study of Foster Carer’s support undertaken with which to compare the results for the FAST survey. This was published by Martin Smith, Hampshire SSD Performance Management Unit in 2002 and appears in the column marked HSSD.
As with all the satisfaction scores, these comparisons are subject variation due to the very small sample size of 9 (5.8% of the target group) whereas the Hampshire study had a sample of 111 (32% of the target group) of whom 55% expressed a satisfaction score. These are presented here for interest and future comparison. Please note that the wording of the Hampshire study was ‘Very Helpful’ rather than ‘Very Satisfied’ etc, as in the IW questionnaires.
The EDS telephone survey, above, has been used as a comparator for the Foster Carer’s satisfaction score with the EDS. This appears in the column marked SURVEY.
|
How satisfied were you
with support from the FAST? |
|||
|
Satisfaction Level |
Numbers |
% |
HSSD |
|
Very Satisfied |
5 |
55.6% |
54% |
|
Satisfied |
3 |
33.3% |
24% |
|
Not Satisfied |
1 |
11.1% |
22% |
|
How satisfied were you with support
from the EDS? |
|||
|
Satisfaction Level |
Numbers |
% |
SURVEY |
|
Very Satisfied |
3 |
33.3% |
100% |
|
Satisfied |
4 |
44.4% |
|
|
Not Satisfied |
2 |
22.2% |
0% |
Analysis. Given the small size of the sample group, great variations have been produced by 1 or 2 respondents; for example, the ‘Not Satisfied’ scores were all caused by 2 Foster Carers having a bad experience in one specific case. The issues that led to these difficulties have been explored and an open invitation has been sent to all Foster Carers to contact the EDS Manager directly should they experience any difficulties in relation to specific incidents ‘out of hours’ in the future.
It is reassuring that the majority of Foster Carers (89% in relation to FAST support and 78% re. EDS) were satisfied with their support. However, the comments made were instructive in highlighting areas for improvement.
Comments. Comments were divided up into two types, case / incident specific and of a more general nature. In relation to both, the EDS Manager has written an article for the Foster Carer’s Newsletter offering some solutions to the highlighted general issues.
Whilst the problems encountered in the one specific case cannot be addressed through the survey process, not least because of anonymity, the newsletter article has offered direct contact and open discussion about any point of concern in the future.
Central to the concerns in one particular case was a full appreciation of the legal constraints within which our services operate. An offer has been made to contribute to training in this area, which was a suggestion of another Foster Carer.
Two Foster Carers said they sometimes had difficulty getting through to the FAST Support Worker’s mobile telephone. This will nearly always be due to reception difficulties. As a result, a ‘default’ process has been confirmed and publicized in the newsletter to ensure that Foster Carers should be able to access support, be that from the FAST or the EDS, within the same timeframe as any other referrer.
Comments were also made that the full protocol regarding ‘out of hours’ support had been printed on the reverse of the questionnaire and that this would have been useful to keep ‘all in one place’. This complaint was answered by the protocol being reproduced as part of the newsletter article with the suggestion that this be added to the Foster Carer Handbook for future reference.
Another comment was that a Foster Carer should automatically be put through to the EDS without referral details. This is one area that cannot be compromised upon, in order to maintain a thorough and safe emergency response some background details will be required in order that the Filtering Officer can briefly research the underlying factors which will influence the emergency intervention. Clear guidance on this was also featured in the newsletter article.
Conclusion. Notwithstanding the small sample group, some useful observations have emerged from this survey of Foster Carer support. Particularly for carers who rarely seek support out of hours, the dual service of FAST and the EDS does need some clarification. It is hoped that the current newsletter article will help address this but ongoing training and, above all, communication is required to resolve this.
As regards the Foster Carer’s use of the EDS, it is hoped that feedback will become more immediate and more regular due to the direct contact on offer. This will allow adjustments to be made, wherever possible, to improve the support available.
Acknowledgements. Thanks are due to: Sarah Pepys, Manager FAST, who helped draft the Foster Carer’s questionnaire and arrange for it’s distribution,
Martin Stevens, Hampshire Social Services, for permission to use his study findings, and Julie Lucas, IW SSHD ICT Section, for her painstaking work in conducting the EDS Service Users Survey and collating it’s results.
Simon Smith. Manager, Emergency Duty Service.
19th September 2003