Low-Cost Pre-Evaluation of New Educational Programs

Tue May 10th 2011

My first industry-track research paper: Low-Cost Pre-Evaluation of New Educational Programs, a paper printed in the proceedings and presented at the international conference in Venice.

The key insight in this paper is a solution to the resource scarcity problem in evaluating educational programs. While the introduction of new programs are meant to improve various learning aspects, in practice, one can never tell how effective a program really is until a "full-fledged study" is done. Conducting such a study requires a lot of resources though, including time, money, and man-power. Since educational programs may last from months to years, evaluating its effectiveness after it has been implemented is especially time consuming. What's worse -- if the program is not effective, one wouldn't have known until it's too late. So what do researchers and practitioners do?

My paper proposes a low-cost solution to this problem. I called it a "pre-evaluation" because it is a purely analytical solution, since one can never really know how effective a program is until it has been carefully carried out. (However, if you can identify the variables involved in measuring effectiveness, you can add them to the analytical solution as well.) Here's the general process:

This solution uses basic utility theory. A nice feature of this solution is that it can be used to compare the effectiveness of multiple programs as well. For example, if three people came to you each with different programs they believe should be implemented, which do you choose?

In the paper, I described the theory, demonstrated how the calculations work, and applied the solution to some field data the team collected. The data is limited in scope and size, but it would be nice to have the team conduct a full fledge study and use those results to compare to the analytical solution. That's future work for now!

Read the paper.


More Blogs

Recovering lost local git commits

A New Client: Thomas Hinds Tobacconist

The Choice Matrix - Official Launch!

A New Client: Manitoba Health

Manitoba Venture Challenge