Category: Evaluation

EVALUATION. It can feel like a heavy word, one with a lot of judgement! But we all evaluate in our everyday lives, even though we don’t always call it that. Evaluation helps us make decisions, determine how well previous decisions are working out, and create data summaries we can share with others to demonstrate the success of our efforts.

The ACT for Youth evaluation team is launching a monthly web meeting all about evaluation. The goal of the series, “Making Evaluation Fun and Pleasurable,” is to demystify evaluation, build your knowledge about and capacity to conduct evaluations, and create a space to talk one-to-one or in small groups about your CAPP, PREP, or SRAE program evaluation.

The first part of each meeting (about 15 minutes) will focus on an evaluation-related topic or skill. The rest of the hour will be open for your questions and/or a time to talk with your TA/evaluation team about YOUR program’s data. Please join us! We’ll meet 11:00 – noon on the 2nd Thursday of each month, starting April 14.

~ Mandy

The Biannual Report (BAR) form is now available through the online reporting system (ORS)! This form requests the same information you sent in during the last reporting period, but it’s in a new format – everything is submitted through the ORS.

A few things to know

• The current report is due January 30 and includes the period July 1 – December 31^, 2017
• Only CAPP/PREP Health Educator Supervisors can access the BAR through the ORS
• Draft descriptive responses in an offline text editor (like Microsoft Word) and then copy/paste your responses into the online BAR

Demos

We’ve created some short videos to demonstrate how to access and complete the BAR through the ORS.

Demo: CAPP Components 1 and 2 – Biannual Report

Demo: CAPP Component 1 Only – Biannual Report

Demo: PREP – Biannual Report

 Have questions?

• For content questions, reach out to your DOH Program Advisor
• For technical questions, contact me at ald17@cornell.edu

  – Mandy

 

With the increasing volume of cycles these past few months, we thought this was a good time to share some reminders about common issues we’ve noticed when reviewing entries in the online reporting system. In this post we clarify exactly what counts as a session, and review the process for adding participants after a cycle is already underway.

What exactly is a session?

A single date is a single session, even if you cover more than one module. Keep in mind that a session is defined by date, but it can include multiple modules and activities. So, even if you cover more than one module, or parts of more than one module, on a single day, it is considered one session.

Why it matters: Showing multiple sessions for a single date will artificially inflate the number of sessions required to complete a cycle. We need accurate information in order to diagnose problems correctly (for example, attendance drop off).

We have modified the ORS so it will no longer allow you to create 2 sessions with the same date in a single cycle.

When and how should a participant be added after a cycle has already started?

Add new participants (those who join after the first session) to the cycle before you record the first session they attended. Keep in mind information should be added to the ORS on a rolling basis; the order in which you enter information should reflect what happened in the cycle.

Why it matters: Entering information about a participant in any other way will negatively affect your attendance rate because the system will not reflect the total number of participants correctly.

To add information about a new participant:

1. Before entering the session data, go to “Update an existing EBP cycle”
2. Select “Add a new participant”
3. Select the cycle the participant should be added to
4. Fill out the demographic information (note: don’t check the “attended” box for the session dates listed, because the participant did not attend those sessions)
5. Hit save

Next go to “Update an existing EBP cycle,” then “Record a session,” and select the cycle.  Now you can enter information about the specific session.

Overall we are very impressed by how well entries into the ORS are going. We just thought we’d send a few reminders before you submit all the cycles started this academic year. Please reach out to your evaluation support team member with any questions!

   – Jenny

We’re excited about how well people have taken to the Online Reporting System (ORS)! At the same time, we know there’s a learning curve, and we’ve been seeing some errors come through. That’s why we ask you to review cycles before you press the “submit” button. To help with the review process, we’ve put together a brief presentation, which you can find here on the ACT for Youth website:

Reviewing Cycles Before Submitting Them into the Online Reporting System (MP4 presentation — we recommend using Firefox, Chrome, or Safari to view)

Slides only (PDF)

Regardless of how you are handling cycle review at your organization, the following things should always be checked (and fixed if need be) before submitting the cycle:

• Have you entered the correct priority population and the correct setting for each cycle?
• Do you have the right number of youth participants entered?
• Do you have youth attendance entered correctly?
• Have all activities been marked as either having been implemented or skipped?
• Did you enter the correct number of sessions and are they marked with the correct date?

Most mistakes you can fix right on the “Review & submit cycle” page. A few things, such as deleting a blank session or blank participant, will need to be changed on the “Revise existing cycle / participants” page.

Adding this final check to your routine will keep your data accurate – and you won’t have to deal with our follow-up calls!

In the future you can find the presentation and slides right on the Online Reporting System information page.

– Dora

We realize that some providers might be lacking infrastructure for fast or wireless internet, and this could be affecting their ability to use the online reporting system and setting up tablets to collect youth survey data. While we can’t do anything about your internet service, we brainstormed some general solutions you can approach your organization’s IT support about.

• Ask your IT support about firewalls or others blocks that might be limiting your access.
• Consider using a tablet with a data plan to enter information in the online reporting system.
• Ask your IT support about setting up a wireless router or adding a wireless access point to boost your connection.
• If hardwired updates aren’t possible, explore the possibility of purchasing a mobile hotspot. Smart phones or tablets with a data plan can often serve as a hotspot, or separate mobile hotspot devices and plans can be purchased.

Once the early kinks are worked out, we are confident you will see the advantages of collecting data online! As always, if you have questions or are having problems with any of our online systems, be sure to let us know.

 – Mandy