Thoughts on Assessment

In the past, I was conflicted on how to perceive assessments results. During my first two years of teaching, I was extremely disappointed with the results that I would receive. Over half of the United States History students received failing grades, some as low as 45 or 50. I discovered through TVAAS, the issue was with me and my role in the classroom rather than the students. The students had no information concerning individual expectations and goals. The lessons and assessment were only sections of lectures formulated into a review day then churned into a test. These test were made from a test generator the department had received from a book company that claimed to be in line with the state standards. I was in survival mode: how I taught, how I assessed and especially how the results were analyzed.
Through implementing different types of assessments, I have challenged myself and the students to think differently about how standards are taught, assessed and analyzed. Now, at the beginning of the year, we as a department embrace data and enjoy breaking down test questions by words and phrases. At the middle of the first nine week period, I see that students are ‘hitting their stride’ or finding their pace to perform on assessments. My goal is to identify one or a group of students who are falling short and reward those who are going beyond their expected level of performance. Rather than using assessments as a 50/50 shot (you fail/ you pass), common assessments, norm referenced, formative and summative assessments shed a light on the students’ true and continuing academic performance.
Common Assessments are Vocabulary Quizzes
In the history department, we implemented common assessment three years after the push of PLCs. The assessment consisted of five questions, multiple choice, that were aligned with and taken from the item sampler online directly correlating with state SPI’s. We picked the questions based upon how often the standard appears on the practiced test. The validity of the assessment shifted to vocabulary issues rather than Historical knowledge. The most popularly missed questions consisted of political cartoons in which caricatures used a word that was beyond the students’ level of comprehension. As a department, we adjusted instructional strategies and addressing content validity by placing key words into lectures and conversation with students. We found that students became self-managing and would genuinely ask for the definition if the word was used in a causal manner. After analyzing common assessments, we mandated that the key words be integrated into formative assessments to measure the students’ performance of application skills. For manageability, we placed key questions of vocabulary focus on the first page of the formative assessment.
Integrating Norm-Referenced and Formative Assessments as a Learning Tool
While analyzing and addressing vocabulary skills, we tracked formative assessment data reliability based on how many questions the student would miss on a forty question assessment. During the period of the students, “hitting their stride”, I monitored the students achievement and compared performance based upon TVAAS projections, then converted the percentage into a baseline for how many questions should be missed on the assessment. Over a period of five weeks, the average student score would deviate from two to eight questions. The goal was for 70% of students to meet or go above their TVAAS projections. The assessments were 70% multiple choice, 30% short answer. For the class as a whole, I identified common questions the students missed on the first few assessments, linked the standard and composed spiraling questions addressing the standards in various ways. After week six, I requested that the students write down how many questions they should miss on an average assessment consisting of 40 questions. Over 80% of the students’ prediction matched or went above the TVAAS projectiles. If the student wrote down a score that was below TVAAS projections, we would have a conference to enable the students to self-monitor their progress. For example: students states they should miss ten and TVASS projections show they should miss six. The student and I set a goal for missing only eight in the next formative assessment, in one month only missing seven. The student became self-directed. In Paul Black’s words, the student has tasted their soup. The system was called a “magic number.” The students were provided clear targets and informed that if they fell below their own projections it would require remediation.
The frequencies of formative assessments were an issue that needed consideration in order to maintain reliability of the assessment findings. The timespan in between assessments was no more than two weeks at the most. If an assessment was delayed by an unforeseen issue, a short formative assessment was implemented with no prior review or forewarning. In order to receive honest data and reflection of student performance, we informed the students that the grading rubric would be more forgiving and stakes would be lowered if they did not meet their “magic number.” This version of an assessment could be an ego stroke or an opportunity to learning from mistakes. It was not a lengthy assessment, and was given in a casual manner. If the students fell short and felt guilt we identified that reaction as empowerment or taking ownership. For the purposes of self-modifying, I requested that students highlight missed standards in their individual notes.
Summative Assessments: The Showdown
Most of the pressure I observed the students placed upon themselves was during the days before a rigorous summative assessment. The stakes were raised and test rigor was based upon how the stage is set for the summative assessment. The main goal was gaining objective and reliable performance documentation. The soup was being delivered to the customer. In rigorous, high stakes norm referenced assessments, the results of positive growth had a substantial impact on increasing interest in subject matter and to improving behavioral patterns in class. The assessments were 100% teacher manufactured with 25% using multiple choice and 75% focusing on short answers. The short answers questions implemented spiral learning techniques with an objective goal of answering the questions using two to three words (if the students mastered a standard via multiple choice, the assessment question would shift to answering a question and applying knowledge in their words). Magic numbers were heavily discussed and student/class goals were provided, for instance: 70% class wide achievement reached equals extra day of enrichment activities.
When the class was motivated with reaching not only individual goals, but class-wide targets, I transitioned from recall and response pass/fail to an empowering learning coach.

Using these techniques, I was able to identify students who were falling short based upon specific SPI’s before any major high stakes assessment, as well as reward students who would not normally be recognized in other classes. Student felt empowered rather than embarrassed of their performance. Students needed, and secretly wanted, structure and feedback that could be provided through these types of communication through assessments. In the past, I handed back graded test and observed reactions of quiet apathy. Now when I given back assessments results, I observed a postive climate of students discussing questions with each other to looking for the item they missed then going into their note book and highlighting the SPI to making statements such as “not next time” or “OK, this is what I need to work on.” I felt that I made an impact by empowering students take ownership of their own targets of academic performance.

Always Adapting,

Joe Collier


A Brief Study In Data and Growth

Scatterplots on the TVAAS website show multiple schools in the middle Tennessee region making significant growth in the end of course assessment despite evidence of economic disadvantages.

What sets these schools apart? Why are they making significant gains when surrounding counties with similar demographics counties are falling short?
Do student factors, such as poverty, correlate with growth and effective learning?
Testing the data:
A teacher questionnaire was developed that broke down subjective and objective factors of potential for higher rates of student growth. The questionnaire was entitled: POET (Potential of Effective Teaching). After developing POET, phone calls were made to teachers within the selection of the schools listed above. A L5 teacher from a middle Tennessee school was chosen as a prime candidate for this model based upon growth in this economically disadvantaged school. The L5 teacher was interviewed and revealed multiple findings ranging from setting high expectations through teacher made essay-based summative assessments, implementing common core models, and establishing a positive classroom environment.
Does poverty play a significant role or barrier to student growth? No. The Sanders and Horne study (the original developers and the value added system) proved this finding as well, but strong personality teachers and student knowledge are not enough to make growth in rural areas of Tennessee. There is a combination of growth obtaining factors that are worth consideration. Where do schools with low growth go from here?  There are multiple barriers to making significant student growth. The largest factor is that of teacher and subject placement.  The next barrier to success is the uncertainty of the upcoming field tests and future state assessments for US History. The L5 educator stated in their interview that they were using graphs, charts and maps in composition books for the past ten years and assessed the students through read and response assessments. The L5 also stated that they were not concerned about methods of state testing or projections as much as setting and maintaining high standards for all students in the classroom.
Recommendations for Administration
Based upon characteristics from effective to novice teachers found in these POET interviews, the following concepts should be considered for schools that are lacking in academic growth.
Refrain from using 100% multiple choice formats for testing, and implement essay and response driven assessments that can provide overarching summaries or themes of state standards rather than focusing a majority of time on specific SPIs.
Develop closer knit PLC environments among freshman social studies teachers with a goal of establishing similar assessment formats, notebooks and especially goal tracking techniques based upon TVAAS projections to set higher expectations. (Model feeder schools)
Establish better lines of communication with the special education department in order to provide quality modifications to achieve success for all students.
Develop more mentoring and coaching, where successful or veteran teachers can help novice or lower scoring teachers
Ponder the ethical and dreadful debate of teacher placement.

Further Research
A range of questions manifest from the research that would be worth consideration for further research. The initial findings on the TVAAS scatterplots showed multiple schools that meet the criteria for high growth in economically disadvantaged schools in Middle Tennessee. Among these schools other factors emerge from involving system wide educational policies to years of average teacher experience. Examples include reward schools, feeder school systems with close knit PLCs, block schedules, Doctorate level educators, and schools have crafted outstanding school homepages that indicate a positive school environment. How would others score in the POET questionnaire? Would a majority of these high growth schools have teacher with high POET scores? Could these high growth schools have teachers with small potential for growth, who are benefiting from school or system policies? What system wide policies do these schools have in common that lead to higher growth versus their neighboring counties?


Always Adapting,

Joe Collier

Student Motivation

Now that the 2015-2016 school year is coming to a close and educators are developing last minute reviews and testing guidance for students prior to summative testing, it is essential to focus on the motivation behind student performance and goal setting. As much as this time could be used to venting about state mandates and test reliability, especially when there have been apparent flaws in the validity of testing this year, let’s turn our focus on the relationship between student, teacher, and the assessment. I firmly believe that the success in education comes from the trust established in the classroom. Co-authors, Anthony S. Bryk and Barbara Schneider wrote a book entitled: Trust in Schools: A Core Resource for Improvement, in this text the central focus of trust comes from the motivations of the principals and teachers and showing in our actions that our motivations are true to bettering schools, teachers, and most importantly the students.

At the beginning of the year, multiple conversations are held in my classroom for the sole purpose of developing this relational trust.  During these conversations, I stress to the students not to let grades be the focus or motivation for academic excellence, but to strive to set their own individual goals for accountability.  The goals come from the student after taking survey tests at the beginning of the year that focus on core reading, writing, and analytical skills.  The students’ set goals are then compared to the state’s projection for proficiency and a majority of the time the student’s goal is higher than the state’s projection. Goals are monitored through the year for remediation and/or enrichment. These goals are more important than the letter grade because they hold the student to a true and tangible standard.

Some students will be motivated by grades whether the motivation comes from themselves, college admissions, or their parents and there is nothing wrong with this, but in general curriculum classes, these examples of motivation might not be as tangible as advanced placement courses. Therefore, teachers must find other forms of motivation through establishing trust and developing the students to set their own levels of success.  The downfall of letting grades be the only motivation is the uncertainty based on the event that the student realizes the flaws in the testing and/or the test will not count for or against them.  If grades have been the only motivation, then what if there is no grade tied into the end-of-the-year assessment. Students lose motivation.

If the student has established their own goal of accountability and the teacher has proven to the student that the teacher’s true motivation is to guide the student to that goal then the teacher has developed relational trust. Weeks prior to the end-of-course, I stress to the student that they study for themselves rather than a state mandate, grades, or even for me as their teacher. When the student goes back home, it will be within themselves to find the reaso and true motivation to open the book or study guide.

Always adapting,

Joe Collier