vignettes/B_quiz_creation.Rmd
B_quiz_creation.RmdThis document describes the steps required to set up a course and create quizzes using quizgrader.
I assume you read the introduction vignette.
Start by loading the package with library(quizgrader).
You can do most tasks related to quiz management through a user
interface (UI). If you want to use the UI, start the main menu with
quizmanager(). This is the interface you will use to make,
manage and analyze the quizzes.
You can alternatively skip the UI and do everything on the command line and by directly interacting with folders and files. The following documentation explains both ways, always starting with the UI approach, then explaining the equivalent direct approach.
In the UI, specify a name for the course, choose a directory in which
you want to place all the folders and files for the new course, and
click on Start new course.
Alternatively, from the command line you can start a new course with
create_course(coursename = "yourcoursename", courselocation = "path/to/location/for/course")You are required to provide a name for your course (which will be the name of the main course directory). It is recommended you provide a location where you want the course folder and its subfolders to go. If you do not provide that, it will be created in the current working directory.
If things go wrong, you should get an error message. Otherwise, there should be a new folder with the course name you specified in the location you chose. You will also get a message telling you where the new folder has been created. Inside that main course folder are several other folders and files, namely:
Folders
/studentlists - a folder that will contains Excel
spreadsheet(s) with student roster information./gradelists - a folder that will contain the main Excel
grade tracking spreadsheets, created by combining the student list and
quiz information (see below for details)./completequizzes - a folder which will contain Excel
files of all your quizzes (with answers) once you made them./studentquizzes - a folder that will contain Excel
files with versions of the quizzes to be given to the students for
filling out and submitting. These files are generated once you made your
solution sheets./studentsubmissions - all individual submissions by
students. Once the grading app is deployed and students start
submitting, their grades will be recorded in the main
gradelist.csv file. For record keeping and detailed
analysis, each individual submission is also kept. For easier
organization, a sub-folder for each quiz will be created and student
submissions organized into those sub-folders./templates - a folder containing templates for quizzes
and student list. Use those as starting points.Files
studentlist_template.xlsx is a template file inside the
/templates folder that should be populated with student
information, as described below.quiz_template.xlsx is a template file for quizzes,
generated inside the /templates folder. This file should be
used as template for creating your own quizzes, as described below.Note that if you retrieve template files through the UI, they
will come from inside the package, not your /templates
folder. This way, if you accidentally mess up the files in the
/templates folder, you can get clean versions
again.
In the UI, you can load a course you created previously, so you can continue working on it. If you are not going through the UI, you will be directly copying files into the course folders. You therefore do not need to set a course folder (but you’ll have to specify the folder when you call the functions).
Once you created the course as described above, you need to add
students to the course. To that end, you need to fill out the provided
template with student information. Get the
studentlist_template.xlsx file either through the UI or you
can find it in the /templates folder on your computer
inside the course folder you just created. The template file has several
columns that you need to fill:
StudentID needs to be a unique identifier for each
student. Students will enter this when they upload their quizzes for
grading, and ut will be checked/matched to identify the student.
Matching is not done on names since those can be non-unique. The best
option is probably to use student emails as ID, but any other unique
identifier works.
The Password column is there to prevent a student from
accidentally or purposefully submitting as another student. Of course,
this requires students to keep their passwords to themselves. This does
not prevent cheating. If students want to, they can share passwords and
submit for each other. Or they can share their filled out quiz sheets
before submission. I expect students to follow the academic honesty
policy, I use lots of low-stake quizzes that count for little of the
total grade (a single quiz usually <1% of the total grade) and most
of my teaching is at the graduate level and students take my courses
because they are interested in the topic. All of these features minimize
the risk of cheating to a level that I don’t worry about it. If
you are worried about cheating on these quizzes, this package/setup is
likely not suitable for your situation!
I collect passwords at the beginning of the course through a simple Google form. I suggest you tell students to only use letters and numbers in their password and nothing else (including no blank spaces). Other characters might work, but in my experience this can lead to occasional problems. I copy the passwords from the Google form/sheet into the template. If you want to, you can likely automate this process with some R scripts. (Or you can contribute this feature to the package, pull-requests with new features are very welcome!).
Disabling the password feature is currently not possible. If you think a setup without a password is a feature I should implement, let me know. For the time being, if you don’t want to use this feature, just enter the same password for every student and tell the students what the password is. They’ll still have to enter it during submission, but that should not be much of a nuisance.
You should tell students that you will be able to read their passwords and that this setup is not very secure, thus they should use a throw-away password for this course and not any passwords they use to protect real systems (e.g. their bank accounts)!
Once you are done filling the student list, add it to the course by
clicking the Add studentlist button in the UI or manually
place it into the studentlists folder. Each student
needs to have a StudentID and Password
entry. Name information is recommended but optional.
During this process, all entries are converted to lowercase and any leading or trailing blank spaces stripped. This is also done to the information students provide when uploading their submissions, thus hopefully minimizing mismatches for silly reasons.
The only time the studentlist file is checked to
ensure it follows the required format is during generation of the
servery deploy package.
If you want to keep track of changes, give the file a new name
(e.g. studentlist_new.xlsx). Any pre-existing file with the
same name will be overwritten. You can also manually copy the file into
this folder. I recommend only using letters, numbers and underscore for
file names. Other names might work, but since the files are deployed to
a server (which is likely running Unix/Linux), having file names that
contain other characters (e.g. dashes or blank) might cause problems.
You can have as many versions of the student list file in the
/studentlists folder as you like. When you generate
the grade tracking sheet (see below), the most recently modified
.xlsx file in this folder will be used. All other
files will be ignored. Make sure this file is a valid student list.
Ideally, at this stage when you are ready to make the
gradelist file, remove all but the current studentlist
file.
To create quizzes, use the provided template. You can get the
template by clicking Get quiz template, or by manually
going into the /templates folder of your course.
For each quiz, you need to create a separate file based on the
template, they should all be structured as shown in the template.
The name of each quiz file needs to agree with the QuizID entry
and be QuizID_complete.xlsx (see next).
Each quiz must have columns with the following names and content. All column names need to be spelled as shown in the template. Additional columns are optional and will be ignored.
A unique ID. This can only contain letters, numbers and underscore.
Only the entry in the first row matters, anything else in further rows
is ignored. It cannot be empty. Quiz files need to have the name
QuizID_complete.xlsx where QuizID is the entry
in the column of that name. So if you set QuizID to
firstquiz then the file name needs to be
firstquiz_complete.xlsx.
A unique ID for each question. This can only contain letters, numbers
and underscore. It is best to give each question a unique ID across all
quizzes for easier question identification during analysis. I recommend
combining quiz ID with some additional question ID
(e.g. quiz1q1, quiz1q2, etc).
The question text. You can use basic formatting, e.g. line breaks and such. Fancy formatting, e.g. equations or pictures or such are currently not supported, and unlikely will be. If you need to provide additional information, you can include a link to a URL into the question, or reference other materials which can contain anything else.
Provide additional instructions to the students, e.g. instructions regarding rounding, entering an integer, etc.
The columns just described will be part of the files that are
generated for the students. An empty Answer column is also
incuded. All other columns described next exist only in your complete
solution files and are not included in the files given to the
students.
The correct answer to the question. Write it exactly the way you want students to specify it. Make sure you don’t have accidental white spaces after your answer. See below for more details on what kinds of answers are implemented.
This is optional feedback that will be provided to the student if
their submission is found to not be correct. The feedback will not be
shown for correct entries. Students will see the feedback after they
submit their quiz, together with the
Correct/Not Correct label. If you want to
allow for more than one submission attempt, you should probably not
provide the answer outright here. If there is only one submission and
you want students to see the correct answers right away, you can copy
the answers into this column.
Type of question/answer. Several types are currently allowed. Depending on the type, answers are evaluated differently, as follows:
Character: A single character is expected and only
the first character is evaluated. Capitalization is ignored. This means
if the answer is A and the student provides A)
or anything that starts with either A or a it
will be marked as correct, otherwise
not correct. Good for multiple choice.
Text: Matches text provided by student with answer
text. Leading and trailing white spaces are trimmed in the submission
and capitalization is ignored, but otherwise matching is strict. Thus,
if the answer is hello world and the student submits
hello world (extra white space), it would be marked as
not correct. So use this type of question/answer with care,
as it is easy for there to be a non-match based on benign reasons,
e.g. a simple typo. Probably best for single words.
Logical: The student is expected to write either
TRUE or FALSE. Also allowed are
Yes and No and 1 and
0. Only the first character is evaluated and capitalization
is ignored. Thus, if a student enters anything starting with
t, y or 1 it will be interpreted
to mean TRUE. Similarly, f, n and
0 are evaluated to mean FALSE. Anything else
is automatically interpreted as an incorrect submission and labeled as
wrong.
Integer: Whatever the student provides is rounded to
the nearest integer, then compared with the answer. So if the answer is
42 and the student provides 41.5 or 42.4, that’s evaluated as correct,
41.4 would be evaluated as wrong. So would 41 or
43. Any submitted answer that can’t be converted to numeric
is labeled as wrong.
Fuzzy_Integer: Similar to Integer,
whatever the student provides is rounded to the nearest integer, then
compared with the answer. The difference is that this type also allows
for a plus/minus 1 difference. This is done in case students don’t know
how to round and you don’t want to penalize them for that. So if the
answer is 42 and the non-rounded answer the student gets is 41.5 and
they round it - wrongly - to 41, it would still be considered correct.
Any submitted answer that can’t be converted to numeric is labeled as
wrong.
When to use Fuzzy_Integer or
Integer: If you know the answer is an integer, e.g. if
the question is “How many US states are there” and the right
answer is 50, you don’t want to allow 49 or 51 to be considered correct,
so you should use the Integer type. If on the other hand
you ask students to do some calculation and the result is 41.50123 and
you want them to round it to an integer, if you want to be strict and
only allow proper rounding (i.e. 42) you should use Integer
type, if you do not want to penalize rounding mistakes, you should use
Fuzzy_Integer which would also accept 41 as correct. A
submission of 40.567 would also be considered correct, since the number
is first rounded to an integer, then checked if it is within plus/minus
1 of the correct answer.
Numeric: An exact numeric value. Submission is directly compared to answer, an exact match expected. This means if the answer is 42.34 and a student submits 42.343, it is labeled as wrong. Any submitted answer that can’t be converted to numeric is labeled as wrong.
Rounded_Numeric: A numeric value rounded to some
specified significant digits. Those digits need to match the provided
answer. Thus if the answer is 42.42, you should specify that students
need to round to 2 significant digits. This again allows for non-proper
rounding, e.g. if a student gets 42.4163 and mistakenly rounds to 42.41
it will still be considered as correct. If you don’t want that and want
to be strict, use the Numeric type.
A note on answer types: I implemented those that I use for my own teaching. If you have others that you want/need, please let me know. Most of them should not be too hard to implement and I’d be happy to do so (if reasonably easy to do).
A date on which the quiz is due. Only the entry in the first row matters, anything else in further rows is ignored. It cannot be empty. It should be coded as a text (don’t allow Excel to transform it into a date). It needs to follow the ISO standard, i.e. be of the form YEAR-MO-DA (e.g. 2021-05-22). Times are currently not implemented, thus students have until midnight of the due date to submit. If you very much want/need a due time, let me know and I can implement that. If you want to have a quiz that has no due date, just set a due date very far into the future.
Number of submission attempts students can make. Only the entry in the first row matters, anything else in further rows is ignored. It needs to be a positive integer. Note that currently, any new submission will overwrite previous ones (which means if a student submits twice and the second attempt is worse, that lower score will be recorded). If you need unlimited submission attempts, set this to a very large number.
I assume you’ll be making your quiz sheets using Excel or similar (e.g. LibreOffice). These programs are sometimes overly eager when it comes to formatting. The safest approach is to format all cells as text. When these quiz sheets are read in, each cell is read as text and then internally converted (to e.g. a date or a numeric value). If you allow the spreadsheet software to do formatting, things might go wrong (e.g. if you read in a date format from Excel as text, it often shows up wrong). Always test your quizzes before giving them to students.
Also note that both Duedate and Attempts columns are not copied over to the student sheets. Instead, I assume that you will provide this information to students in some other way, e.g., when they receive their copies of the quiz files to be filled, or posted somewhere in your syllabus or other course materials.
Once you created your quizzes, you need to place them into the
completequizzes folder. To do so, use the
Add completed quiz to course button in UI. You need to add
one quiz at a time. The quiz will be checked for proper content and a
file with the name of the QuizID, appended by _complete
will be placed in the completequizzes folder.
You can also add completed quizzes manually to the
completequizzes folder. If you do so, files are not checked
for correctness. They will be checked during creation of student sheets
and the grade sheet. You also need to ensure the files have the right
names. Complete quiz files need to be named
QuizID_complete.xlsx where QuizID is the entry
in the column of that name inside the quiz sheet.
Once you are done creating quizzes, you should have a collection of
properly formatted Excel files in the completequizzes
folder. Remove any files from that folder that are not quizzes you want
to use. All your quizzes need to follow the naming convention as
described previously. To create quiz sheets that will be given to
students, click the Create student quiz files button in the
GUI, or run:
create_studentquizzes(courselocation = "path/to/location/for/course")This checks each quiz file in the completequizzes folder
for proper formatting and content, and if successful, generates a
matching sheet to be given to students. The student sheets will contain
the columns QuizID, QuestionID,
Question, Instruction and an empty
Answer column. The remaining columns will not be included
in the student sheets.
You can find all the student quiz files in the
studentquizzes folder. There is also a zip file that
includes all quizzes. You can get this file by clicking the
Get zip file with all student quizzes button in the UI (or
as always by going directly into the studentquizzes
folder.)
Distribute these files to the students by any means you like
(e.g. email, Dropbox, posting to a course website, etc.). Students will
fill out the quiz sheets by entering their answers into the
Answer column, and then submit them through the online
submission app.
If you update/edit/add quizzes, make sure all updated files are in
the completequizzes folder, then re-generate the student
quizzes and re-distribute.
Note that during the process of student quiz creation, the
studentquizzes folder and all its content are deleted and
newly created - so don’t store anything in there!
At this stage, you should have a filled student list in the
studentlists folder and have all your quizzes done and
stored in completequizzes (and generated equivalent student
quiz sheets). I recommend only having your most current student list
file in the studentlists folder and only the quizzes you
want to use in the completequizzes folder.
In this step, you will generate the main file that will track all
student grades. This will merge information from the student list and
all quiz sheets. To generate this main tracking file, click the
Create grade tracking list button or run this command:
create_gradelist(courselocation = "path/to/location/for/course")If you use the UI, this will also run the student quiz generation routine to ensure everything is in sync. If you don’t use the UI, make sure that your student quiz files match the current version of the quizzes.
Both the student list and each quiz will be checked for proper
formatting/content. Fix any errors you might get. If everything works,
the grade tracking list will be placed into the gradelists
folder. It will have the name
DATETIME_gradelist_COURSENAME. Note that the quizzes are
ordered in this tracking document based on their file names and not
their due dates. If you want to have the quizzes show up in the order in
which they are given in class (because you might want to manually look
into the grade tracking sheet), you need to make sure the file names of
the quizzes follow the order of the due dates.
If the student roster changes, or if you modify/add quizzes, before students have started to submit quiz solutions, you can always re-create a new tracking document. Once students have started submitting quizzes, things get trickier. Managing changes to class list or quizzes in that setting is explained in a later tutorial.
Once you created the class list and all quizzes, produced quiz sheets for students and generated the main grade tracking document, you are ready to deploy the student-facing part of the quiz system. Those steps involve:
Provide students with the quiz files they need to fill out.
Provide students with any additional information they need for the quizzes (deadlines, allowed submission attempts, website for submission, etc.).
Deploy the grading app to a Shiny server and tell the students where to go to submit their completed quizzes.
Once the grading app is fully deployed, it works as follows:
Students fill out the quiz sheets, the go to the URL where your app lives, enter their information and upload their quiz.
The online system grades the submission by checking it against
the solution. It provides immediate feedback to students on how they
did, and records the grade in the main tracking sheet. All student
submitted files are saved in the studentsubmissions folder,
which allows for detailed analytics and provides a complete track record
of any submission (in case something goes wrong in the main grade
tracking file).
You occasionally retrieve the grade tracking sheet and the
student submissions from the server and use the
Analyze Quizzes tab in the UI (or equivalent functions on
the command line) to see how students are doing, and to check which
questions are problematic and indicate students didn’t fully understand
that content (or you wrote a bad question).