Syllabus

Assessing Measures and Measuring Assessments

Measurement is a principle trait and whether it is hard wired, learned, or some combination of both, its absence makes quantitative data impossible to classify. Even in cases where measures are available, how can we know if they are consistent and accurate? Without these checks, instruments such as surveys could never be constructed, much less yield results that could be generalized.

When assessing existing programs, having an ability to test a hypothesis is key in understanding its goals. While evaluation tells us whether a program is producing results or having an effect/impact, (performance) measurement tells us what a program did and how well it did it. In particular, the latter is a necessity and involves collecting and reporting data that can be used to compartmentalize the way a program is being implemented.

Course Objectives

The overall purpose of this course is to empower you to

• understand the connect data and measurement principles

• be skeptical of every research question or purpose that you come across

By the end of the course you should ideally be able to

• address differing types of measures and variables
  

• determine the validity and reliability of tests

• derive measures from items

• interpret and report psychometric properties of assessments

Additionally, this class will expose you to — one of the most popular and in-demand statistical languages. Armed with the foundation of R skills you’ll learn in this class, you’ll know enough to be able to find how to analyze and visualize many outputs based on measures.

Overview

Move back and forth using ← and →

Toggle fullscreen by pressing F

Press O for and overview of all slides

Discover more shortcuts by pressing H

Larger version of the Overview	PDF of the Overview

FAQ

Is the course content difficult?

You’ve probably heard an answer like this before: At times some of the material can be dense. Well that isn’t much of a response. Difficulty is not the issue here since as humans with differing educational background, we will have strengths in some areas more than others. Instead the question is can you identify areas that need strengthened and communicate them? If you can, then you have a good shot at succeeding in this class.

Is R difficult?

Learning can be especially challenging at first—it’s akin to learning a new language like Spanish or even mathematics. Even experienced R users get frustrated…and so much so that some of us have swear jars. However as silly as it sounds one of the best feelings is to overcome a roadblock. With that said, if you find yourself getting irritated, try the following: take a break, go let some frustration out, sleep, discuss with a peer, etc. If you are at your limit, take a few breaths and contact me!

What if I’m a Visual/Aural/Read-write/Kinesthetic learner?

You’re not one or even any of those. There’s not one accepted study that supports one type of learning style over another. In fact, there aren’t even any studies that support these four classifications either. You learn the best using multiple approaches which is why there is both a Readings and Extras section. At one time I denoted multiple papers to support this, but many of the sources I listed were also cited in this fantastic video:

What if I find a mistake?

Tell me! I strive to be error free but unfortunately like everyone else, make silly mistakes including grammar and spelling errors as well!

Materials

There are four optional texts and two software packages necessary for this course. With that said, you will receive some supplementary materials in the course as well.

Texts

Optional Reference

To help minimize costs, there is no formal text. We’ll rely on the text below which is available through the WVU library WVU library in an online capacity barring the APA 7th edition handbook.

• Balkin, R. S., & Kleist, D. M. (2016). Counseling Research: a Practitioner-Scholar Approach. American Counseling Association.
  • Free: Digital edition
  • Not Free: Softcover edition - ISBN: 9781556203572
    
    
• Albano, T. (2020). Introduction to Educational and Psychological Measurement Using R.
  • Free: https://thetaminusb.com/intro-measurement-r/
    
    
• American Psychological Association. (2020). Publication manual of the American Psychological Association (7th ed.).
  
  
• Wickham, H., Navarro, D. & Pedersen, T.L. (2021). ggplot2: Elegant Graphics for Data Analysis (2nd and 3rd eds.). Springer.
  • Free: Digital edition
  • Not Free: Hardcover edition - ISBN: 9783319242750
    
    
• Wickham, H. (2021). R for Data Science (1st ed.). O’Reilly Media.
  • Free: Digital edition (web)
  • Free: Digital edition (WVU libraries)
  • Not Free: Hardcover edition - ISBN: 9781491910399

There will occasionally be additional articles and videos to read and watch. When this happens, links to these other resources will be included on the content page for that session.

Software

Quatrics

You will need access to your WVU Qualtrics account. Please go to the site and check if you can login. If you are enrolled as a WVU student, then you should have access. However if you do not, submit a ticket to IT or give them a ring at 304-293-4444.

R and RStudio

You will be briefly introduced to the open source (and free!) statistical programming language later in the term. You will use RStudio as the main program to access the language. Think of R as an engine and RStudio as a car dashboard — R handles all the calculations produces the actual statistics and graphical output, while RStudio provides a nice interface for running R code. Please note that

You can find instructions for installing R and RStudio here.

Help

Data science and statistical programming can be challenging. Computers are stupid and its always the tiny errors in your coding can result in tons of headache. People working in any syntax based software package at any level experience this!

But there are multiple resources both online and locally that are available. Prior to our first session, please take a moment to go through this slideshow for a short description of the main outlets.

Move back and forth using ← and →

Toggle fullscreen by pressing F

Press O for and overview of all slides

Discover more shortcuts by pressing H

Larger version of the presentation	PDF of the helpfile

Assignments and Grades

You can find descriptions for all the assignments on the tasks page.

Percent	Assignment	Location
15	R Training	Data Camp
15	Reflections	Slack
15	R Measurement EDA	eCampus
15	Check-ins	Zoom
30	Proposal	eCampus/Slack
10	Elevator Pitch	eCampus/Slack

Grade	Range
A	90 – 100%
B	80 – 89%
C	70 – 79%
D	60 – 69%
F	< 60%

Office Hours

Please watch this video:

Office hours are set times dedicated to you the student!¹). This means that I will be in my office or at home waiting for you to come by talk to me remotely with whatever questions you have. This is the best and easiest way to find me and the best chance for discussing class material and concerns.

Due to the ongoing pandemic, we can only meet via Zoom. This can be scheduled using the calendar app.

Conduct and Expectations

Here are the rules, expectations, and policies that I came up with or stole from other professors:

• Late work: Barring the in-class only tasks, past due deliverable will only be accepted up to 48 hours after the initial time and due date. For each full day an assignment is late, 10% of the final grade will be deducted. All submissions must be made via eCampus. There are no exceptions to this policy. Please note that coursework will not be accepted by email or any other means.

• Participation: Please ensure that you are engaged and participate in class. Engagement is mostly defined by you — if that means commenting and answering questions - great; if it means sitting quietly and being focused - also great; but if it means being being disrespectful or flaking off - not so great.

• Justifications: On any submission, you must justify any assertion. I don’t know what you know so its your job to provide all of the necessary evidence to convince me that you do know what you say you know. While you’ve probably heard this multiple times over your life, think of it this way: if someone tells you that the core of the Earth is made of pixy sticks, the first question you should be asking is what’s your evidence?.

• Rubrics: While there are valid reasons for the utilization of a rubric in undergraduate classes, at the graduate level, I do not (often) provide nor use a such an item to guide or evaluate your submission due to four primary concerns:
  

  1. When writing anything in academia that is pivotal (a thesis, dissertation, journal article, report, etc.), a rubric isn’t often provided.
  2. If you write within the limitations as defined in a rubric, then creativity may be stifled (i.e. writing to the rubric rather than constructing a product from the ground up).
  3. Feedback can only be given along the criteria listed within a rubric which limits your learning as a student and constrains me as the instructor.
  4. Unless you are in a very specific area, the real world does not use rubrics!
     
     

• Technology use: Use phones, computers, etc. responsibly. We’re all adults².

• Assumptions:
  

  1. Just assume that all submissions are to be written formally and should include with the appropriate use of language, grammar, syntax, etc. and follow standard APA 7th edition formatting guidelines where applicable.
  2. People who are easily offended by content, believe their work to be flawless or are generally unable to handle criticism should consider looking at another course. If you want rainbows and ponies, consider another class. If you care about data, fieldwork and learning a highly marketable practice, you’ve come to the right place.
  3. There is a great deal of content in this course and you will likely struggle with some at times. Given that, there is also something to be said about the satisfaction a person gets when figuring something out, but nowhere is it written that has to be on your own. You may find that a nudge here or there elicits the same feeling so please reach out for help.

Learning and knowledge during a pandemic

When course objectives are written explicitly and clearly, they provide the information you need to figure out what a student should be able to do by the end of a given term. In fact, professors often test your proficiency in an area through multiple assessments such as exams, papers, presentations, etc where you are essentially asked to show us what you have learned. However learning is not the same as knowledge . To save you from a long philosophical narrative on epistemology, in a nutshell we humans aren’t that good at evaluating a person’s knowledge mainly because its not a well-defined concept. With that said, an indicator of knowledge is in a person’s ability to successfully explain a high level concept in such a way that the lay person can understand it. Every so often, consider asking yourself this:

Can I describe whatever using language so that a child could understand it?

Course Policies

In a nutshell keep an open line of communication, be honest, nice, & respectful to everyone.

Additially please follow WVU’s Code of Conduct.

This syllabus reflects a plan for the term but things change and plans change. so deviations may become necessary as we move along during the term. Note that I reserve the right to alter or amend this syllabus.

COVID-19 statement

WVU is committed to maintaining a safe learning environment for all students, faculty, and staff. Should campus operations change because of health concerns related to the COVID-19 pandemic, it is possible that this course will move to a fully online delivery format. If that occurs, students will be advised of technical and/or equipment requirements, including remote proctoring software.

In a face-to-face environment, our commitment to safety requires students, staff, and instructors to observe the social distancing and personal protective equipment (PPE) guidelines set by the University at all times. While in class, students will sit in assigned seats when applicable and wear the required PPE. Should a student forget to bring the required PPE, PPE will be available in the building for students to acquire. Students who fail to comply will be dismissed from the classroom for the class period and may be referred to the Office of Student Conduct for further sanctions.

If a student becxomes sick or is required to quarantine during the semester, they should notify the instructor. The student should work with the instructor to develop a plan to receive the necessary course content, activities, and assessments to complete the course learning outcomes.

Psychological and Psychiatric Services

Life at WVU can be complicated and challenging, especially during a pandemic! You might feel overwhelmed, experience anxiety or depression, or struggle with relationships or family responsibilities. Psychological and Psychiatric Services provides free, confidential support for students who are struggling with mental health and emotional challenges. The office is staffed by professional counselors and psychiatrists who are attuned to the needs of all types of college and professional students. Please do not hesitate to contact them for assistance—getting help is a smart and courageous thing to do.

CARE Team

If you or anyone you know may be at-risk such as those listed here, please make a CARE referral. You may do so directly at the main WVU CARE TEAM site.

Lauren’s Promise

Lauren McCluskey, a 21-year-old honors student athlete, was murdered on October 22, 2018 by a man she briefly dated on the University of Utah campus. We must all take action to ensure that this never happens again.

If you are in immediate danger, call 911 or the Campus Police at 304-293-3136.

If you are experiencing sexual assault, domestic violence, or stalking, please report it to me and I will connect you to resources or call/text a private Title IX On-Call Line 304-906-9930.

Any form of sexual harassment or violence will not be excused or tolerated at West Virginia University. WVU has instituted procedures to respond to violations of these laws and standards, programs aimed at the prevention of such conduct, and intervention on behalf of the victims.

Academic Integrity

The integrity of the classes offered by any academic institution solidifies the foundation of its mission and cannot be sacrificed to expediency, ignorance, or blatant fraud. Therefore, I will enforce rigorous standards of academic integrity in all aspects and assignments of this course. For the detailed policy of West Virginia University regarding the definitions of acts considered to fall under academic dishonesty and possible ensuing sanctions, please see the West Virginia University Academic Catalog at http://catalog.wvu.edu/undergraduate/coursecreditstermsclassification/#academicintegritytext. Should you have any questions about possibly improper research citations or references, or any other activity that may be interpreted as an attempt at academic dishonesty, please see me before the assignment is due to discuss the matter.

Inclusivity Statement

The West Virginia University community is committed to creating and fostering a positive learning and working environment based on open communication, mutual respect, and inclusion.

If you are a person with a disability and anticipate needing any type of accommodation in order to participate in this class, please advise me and make appropriate arrangements with the Office of Accessibility Services ( 304-293-6700).

For more information on West Virginia University’s Diversity, Equity, and Inclusion initiatives, please see http://diversity.wvu.edu.

Incomplete Grades

Students who want to be considered for an Incomplete must apply to their instructor prior to the end of the term. If the instructor agrees, the instructor and the student must negotiate the conditions under which the grade of I will be changed to a letter grade and sign a contract. The date to submit the incomplete work should not be set beyond the last day of class of the following semester. If the student does not complete the terms of contract, then the instructor should submit a grade of F. All incomplete contracts must be filed with the department and Dean’s Office. See the policy at [Students who want to be considered for an Incomplete must apply to their instructor prior to the end of the term. If the instructor agrees, the instructor and the student must negotiate the conditions under which the grade of I will be changed to a letter grade and sign a contract. The date to submit the incomplete work should not be set beyond the last day of class of the following semester. If the student does not complete the terms of contract, then the instructor should submit a grade of F. All incomplete contracts must be filed with the department and Dean’s Office. See the policy at http://catalog.wvu.edu/undergraduate/enrollmentandregistration/#gradestext.

Sale of Course Materials

All course materials, including lectures, class notes, quizzes, exams, handouts, presentations, and other materials provided to students for this course are protected under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. Please review the sharing and editing restrictions prior to distributing or amending any material on this site. As such, the unauthorized purchase or sale of these materials may result in disciplinary sanctions under the Campus Student Code. Basically you can share what you like but don’t try to make a buck.

Student Evaluation of Instruction (SEI)

Effective teaching is a primary mission of West Virginia University. Student evaluation of instruction provides the university and the instructor with feedback about your experiences in the course for review and course improvement. Your participation in the evaluation of course instruction is both strongly encouraged and highly valued. Results are strictly confidential, anonymous, and not available to the instructor until after final grades are released by Admissions and Records. Information about how you can complete this evaluation will be provided later.

University Attendance Policy

At West Virginia University, class attendance contributes significantly to academic success. Students who attend classes regularly tend to earn higher grades and have higher passing rates in courses. Excessive absences may jeopardize students’ grades or even their ability to continue in their courses. There is a strong correlation between regular class attendance and academic success.

Course Netiquette

The basic premise is that the etiquette expected of students in the online environment is the same as that expected in a classroom. Common courtesy is the guiding rule of Internet communications. Be prepared to communicate effectively when taking an online course. Following these simple netiquette rules in your online class or education environment will ensure your success:

• Include a professional salutation. In this case, “Hello Dr. Roy” or “Dear Dr. Roy” is appropriate.
• Include a proper ending such as “Thank you” or “With regards.” Then type in your full name.
• Never type in ALL CAPS, because it reads as if you ARE SHOUTING AT PEOPLE.
• Act as professionally, via your writing, as you would in a face to face classroom.
• Refrain from inappropriate language and derogatory or personal attacks.
• Do not dominate any discussion. Give other students the opportunity to join in the discussion.
• Disagree with ideas but avoid challenges that may be interpreted as a personal attack.
• Check that you are replying to the specific person you intend, and not to the entire class.
• Never give your password to another person.
• Respect the virtual classroom. Never forward in-class communications or posts by others outside of this virtual space.
• Never spam your classmates.
• If you quote someone’s previous post, only quote enough to make your point.

Be aware of the University’s Academic Integrity and Dishonesty Policy http://catalog.wvu.edu/undergraduate/coursecreditstermsclassification/#academicintegritytext. You can review the rules, regulations, and procedures concerning student conduct and discipline for the main campus of West Virginia University, at http://campuslife.wvu.edu/r/download/1802350.

Response Time

I generally respond to Slack queries in the same day while responses to emails and discussion posts are within 48 hours, except during holidays. Often, I will reply much more quickly but you should not count on a immediate. Please plan accordingly so that you don’t miss deadlines! I generally return assignments within one to two weeks after a final submission date.

Technical Requirements

Students need to have access to a computer for word processing, e-mail and access to eCampus. Access to the Internet is necessary for completion of this course. Run the Browser Check. This tool will check that you are using a supported Internet browser and have a valid Java version installed. The required technical skills to participate in this course are:

1. Navigate the web
2. Use email with attachments
3. Create and submit files in commonly used word processing program formats
4. Copy and paste
5. Download and install software
6. Consult software tutorials and other online sources as a method of learning software features
7. Use syntax when necessary

Notice that programming is not on here!

Technical Support

Technical support regarding your use of eCampus is available by contacting 304-293-4444 (telephone), 1-877-327-9260 (toll free number), itshelp@mail.wvu.edu (email), and/or http://it.wvu.edu (website).

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.