Category Archives: Personalized Learning

ePortfolios in Undergraduate Biology Research: Transcribing Data, Translating Learning

The Central Dogma of Molecular Biology states that cellular information is stored in DNA, converted into mRNA to leave the nucleus, and finally turned into proteins. These processes make sense of the stored information in order to do the actual work of the cell. All the while the cell is sending and receiving signals from both its internal and external environment that drive these processes.

Learning in the biology laboratory can be likened to this in that our students will rely on their knowledge, knowledge gained from the scientific literature, and guidance from their mentor to generate, analyze, and place their results into the bigger picture driving scientific discovery. Novel questions are addressed through the process of the scientific method – a recursive experience with no finish, only more questions. Communication between mentor and student is critical. Communication between the lab and the greater scientific community is also critical. Often overlooked, though, is the necessary communication between the student and themselves. Making sure these dialogs are rich and meaningful can be a challenge when research takes place at the undergraduate level.

How can we facilitate this multi-modal communication to develop critically thinking experiential learners that work towards solutions for the problems they have set out to solve?

                                    Enter the e-portfolio.

Mentors communicate with students

Our student researchers spend only a short time, when compared to students in graduate programs, investigating research questions in our laboratories. Their energies are also spread out in many ways: classes, athletics, and oftentimes jobs. As such their ability to grow in the lab setting is less than if their focus were solely on research full time. However, for the undergraduate student who does not conduct research full time there are less obvious opportunities to interact and provide regular feedback. E-mail is helpful, but not an ideal strategy. Specifically, there needs to be an ongoing dialogue with each student about the progress and pitfalls of their project and email is not setup to meet this need. Through e-portfolios we do have the ability to have regular dialogue with students and the technology can be set to alert us to any student initiated changes, generally the addition of questions, data tables or figures. With this centralized site we are able to review their work in a timely fashion and not have to sort through clogged email inboxes to figure out where the information is. Students progress and benefit as we can catch errors in procedure early and correct them. Posting questions to students has also facilitated a deeper understanding on their part as they can reflect on our questions prior to answering when asked ‘on the spot’ in the lab.

Students talk to themselves

The role metacognition, or the awareness and analysis of one’s own thinking, plays in deepening student learning is well-established. A simple exercise in self-reflection can be implemented into the undergraduate research experience to deepen students’ conversations with themselves about what they know and don’t know. Through regular reflection and posting to the e-portfolio, research students can visualize a time-stamped progression of their learning. They can more easily identify the missing pieces and work to determine why they think what they think. In-turn, this leads to more meaningful interpretations of data, linking their findings to the big picture, and articulating to their mentor (knowingly or unknowingly) gaps in understanding making it easier for mentors to mentor. Students’ critical reflections are essentially conversations with themselves and demonstrate the learning experience unique to each student. This experience, in itself, better prepares students to have these same conversations with others (e.g. in interviews or future collaborative work).

Archive and pass the baton

In research labs students are taught to keep all of their data in a laboratory notebook. This is ideal to ensure nothing is lost, but challenging to sort through after students have left and we need a specific image or data table that we think a particular student generated. With the e-portfolio we can encourage them to archive the ideal images and final versions of data tables that are based on what is added to a notebook on a day to day basis. This obviously facilitates data retrieval after the fact, which is the norm with the pace of data collection. Current students are supplementing or creating protocols that will live on the e-portfolio site allowing each future user access from any internet portal. This is ideal as we can lock down editing on a master version and have students work from a copied version on their portion of the site. This allows us to preserve an original and note additions and deletions added for individual projects.

Benefits to current students may be obvious, but to the next student we think they are even greater. When we open access to the site to a new student they are given existing protocols and can see how data should be archived thus reducing the learning curve and hopefully ensuring more productivity than in the past. This ‘passing of the baton’ from one student to the next also ensures we have an obvious record of who contributed what to any one project.

Share with others

Whether our research students are building upon a shared e-portfolio for the laboratory, or generating their own unique sites, they are exercising their communication skills. They will have virtual conversations with us as their mentors that allow for deeper learning, more rapid identification of “teachable moments”, and self-correction. The use of the e-portfolios is also ideal for future considerations about authorship, or acknowledgements, in publications. Finally, the records are permanent scholarly showcases that Fisher students can maintain for free even after graduation.

Links to Biology students’ e-portfolios

These students have given permission for us to share their work.

https://sites.google.com/a/sjfc.edu/best-practices-in-mentoring-and-assessing-ug-research/showcase/extra-curricular

Written by Kristin Picardo and Ed Freeman, Biology Department

 

Advertisements

What is a Smartpen?

At St. John Fisher College, Office of Academic Affairs supports students with disabilities by loaning Livescribe Smartpens. The smartpens enable users to capture, search, and share handwritten notes. The smartpens synchronize handwritten notes with recorded audio. Currently we have three models available for loan; Echo, Sky Wifi, and Livescribe 3 smartpens.

How do smartpens help students with learning disabilities?

  • Note-taking help, homework help and before a big exam
  • Tap your notes and the smartpen will play back the professors explanation word for word
  • Organize your notes; play back controls allow you to slow down or speed up the audio recording, even bookmark key information
  • Capture everything you hear and write – be confident never miss a word
  • Capture words, scribbles and diagrams & syncs everything to what is said

Not all professors allow students to use tablets or mobile devices during class, students take notes via pen and paper.  It is still the most popular method of note-taking. Shocking but true. Reading your notes and/or listening to your recordings within a web based application such as Evernote allows users flexibility and control when reviewing what was captured, making the Livescribe pen a useful tool for today’s digital lifestyle.

The smartpens have an embedded infrared camera that detects pen strokes on special Livescribe paper. Below are my reflections on the 3 models and best use practices. More detailed information can be found at livescribe.com.

The newest smartpen is the Livescribe 3, this pen uses bluetooth technology; notes appear on your tablet or smartphone instantly when paired with the Livescribe+ mobile app.  Your notes are organized, tagged, searchable and can be converted to text.  Turns your words into action!

smartpen3Livescribe 3 smartpen

To transfer your handwritten notes to Livescribe+, connect your Livescribe 3 smartpen to an iOS device that is Bluetooth ready.

  • iPhone 4S or newer
  • iPad 3rd generation or newer
  • iPod touch 5th generation or newer

Notes:  Need iOS device and Livescribe+ app (free), additionally if your iOS device is NOT bluetooth ready, you can still use the Livescribe+ app to open and review pencasts that are send/shared with you by other Livescribe+ users.

Ease of use/setup:

  1. Download the free Livescribe+ app – via App store designed for both iPhone and iPad
  2. First time, pair device.
    • iOS device- ensure Bluetooth is on
    • Turn on your smartpen, by twisting the middle ring clockwise
    • Open Livescribe+ app – device automatically detects your smartpen
    • Tap/touch Pair when prompted, after it connects the LED on smartpen turns blue and  smartpen icon appears top right corner of Livescribe+ app.

smartpen3onFeatures of Livescribe+ app:

  1. The app can be installed on multiple devices- up to four.
  2. Real-time transfer of notes to supported devices.
  3. Enable sign-in and send notes to an existing Evernote account, as well as OneNote.
  4. Send each page or snippet as an image file, or PDF.
  5. Notes can be sent directly to your OneNote Notebook and/or Evernote app.
  6. Notes taken while audio is recording appear in green on your pages and you can play the recording while in Pencasts view.
  7. Rename, delete, and share pages of your Notebooks.
  8. Add content to handwritten notes, add photos, text, and audio as well as converting handwriting to text. Swipe from left to right- reveal the converted text.

My observations:  At first review of this product I was extremely excited not having to be dependent on Wifi and the ability to sync my notes to OneNote.  The students ( I worked with) who wanted to try the newest smartpen used their cell phone- none had an iPad.  I use this pen for meetings, conferences, and other business situations during my work day.  If you are asked to be a note taker – the smartpen can be a life saver!

The Sky WiFi smartpen records everything you write and hear using WiFi technology.  This smartpen can also connect to the internet through your computer and the Livescribe Helper application. This is an alternative for updating and synchronizing your recorded notes and audio with Evernote. Your words, your ideas, any time, anywhere!

skypenSky Wifi Smartpen

It takes 4 simple steps to get started:

  1. Create a Livescribe account- it is free  www.livescribe.com/setup
  2. Link and/or create an Evernote account – its free and need to authorize (link Evernote and Livescribe accounts)
    Both of the steps above need to be done before you can begin to use your Sky Wifi pen.
  3. Activate your smartpen- enter the characters displayed on the screen of the smartpen and connect to WiFi.
  4. Download and Launch Livescribe Helpter – I use the helper app to update the firmware on the smartpen.  If you don’t have access to WiFi this app is a convenient way to backup/synchronize your notes and audio.

helperRecording audio using the embedded microphone for smaller recording environments such as classroom or conference room.  For larger lecture hall, use the Livescribe 3-D Recording Headset, which contains a microphone in each earbud.  I recommended that students sit close to the front of the classroom to ensure that the audio is captured clearly.

The Echo smartpen uses USB technology to transfer notes from smartpen along with Livescribe Desktop software.  Notes can be shared to Evernote and exported in PDF format.  Capture it, Replay it, and Send it!!

echoEcho Smartpen

Get started by:

  1. Download Livescribe Desktop software – available for Windows or Mac OS.
  2. Connect the Echo using the USB cable to your computer. Connecting the smartpen will transfer your notes and audio to Livescribe Desktop automatically.   Don’t disconnect until transfer is complete.
  3. Register and rename smartpen.

My Observations:  I found this pen more time consuming when loaning out to students, depending on the age of their computer, installing the software can take time.  The Echo smartpen ties the student to their desktop/laptop to transfer/sync notes & audio files.  I have students who prefer this model, but struggle when their computer isn’t working or when they purchase new a computer and transferring notebooks/files.

What do all the smartpens have in common?

  1. Sync and transfer notes from smartpen
  2. Use the Livescribe Dot paper
  3. Search handwritten notes
  4. Replay audio from devices
  5. Direct sharing to Evernote
  6. Send/Share via email
  7. Export audio files
  8. A great tool for everyone- no student should go to college without one!  (my opinion)

Livescribe has a great Comparison Chart for you to reference/view all 3 smartpen features side by side. – See more at: http://techtips-rs.blogspot.com/#sthash.P2rrHoX3.dpuf

Resources:

Livescribe Smartpens. (2014, October 17). Retrieved from Livescribe, Inc.: http://www.livescribe.com/en-us/smartpen/

ePortfolios: Benefits, Tips and Ideas for the Future

Like many academics, I have a healthy skepticism of assessment. As a social scientist, I usually object on methodological grounds. In particular, assessment instruments often focus on immediate or short-term gains and do not account for deeper and longer-term learning. Despite my lack of enthusiasm for assessment, I have come to appreciate the increased intentionality that comes from participating in formal assessment. Thinking about assessment increases the amount of thinking I put into assignment and course creation. In other words, assessment usually produces shallow understandings of student learning but it does make me a more effective teacher.

In most courses, most of us typically assess a small sample of students on a few assignments. While this is very useful for thinking about the content and approach to a specific course, this does not really speak to questions about durable learning, i.e., the acquisition of skills across an undergraduate career or throughout a course of study. E-Portfolios offer a remedy to this problem.

This platform, available to all Fisher students via access to Google sites, allows students to archive and reflect on their entire collection of undergraduate work. The space also allows students to establish a professional web presence that may be of use as they apply for graduate school or enter the labor force. The Google E-Portfolio offers a number of advantages that can benefit the student and the faculty member, including:

  • It is intuitive, free, and easy to customize.
  • They platform is portable and students will retain control over the context after they graduate.
  • The College has developed a generic template that provides a useful structure for organizing student work. This template has built in categories that provide spaces for students to upload files and engage in meaningful reflection on the value of that work for designated requirements, i.e., the first-year program, the core, etc.
The_Core_-_SJFC_ePortfolio_Template

Click image for larger view

 

  • Individual departments may also amend or expand the generic template to serve their own needs (more on this below).
  • Students control the privacy settings that range from closed, i.e., not visible to anyone other than the student, all the way to publically visible to anyone.

I have been using the E-Portfolios in most of my courses for a few semesters, including major courses, core courses, and courses in the first year program. The results have been uneven but are improving. Below, I have listed some tips that I believe enhance the quality of the portfolios and stimulate more metacognition through directed reflection.

  • Introduce the E-Portfolio early in the term and walk the students through setting up it up using the computer and projector. Reinforce this by distributing the link to the online help (http://www.sjfc.edu/campus-services/ed-tech/technologies/eportfolio/).
  • Show students an example of a complete and successful E-Portfolio. This can help students develop a sense of buy-in. Have an in-class discussion on how a successful and professional E-Portfolio might help the student, e.g., providing a professional showcase for their work that might help with graduate school admissions or landing a job.
  • If you plan to use the space as a means to collect written work (or plan on grading the E-Portfolio, including student reflection) you should spend additional time on the privacy settings. In my courses, I require all students to set the privacy settings so that any member of the Fisher community can see their portfolio. This limited visibility serves several purposes. It allows other assessors, e.g., the Core Committee, to review the work. In addition, by opening the E-Portfolio to campus visibility it creates some incentive for students to take the work seriously.
  • Require students to post the artifact and author reflection as part of the grade. This, too, helps create an incentive for students to think more carefully about creating a professional looking E-Portfolio. In the first few term that I used the E-Portfolio, I required that students post the artifacts and author reflection by the end of the term. I quickly realized that this was a bad approach since most of the students waited until the very late minute and the quality of their E-Portfolio suffered as a result.

All of the preceding commentary is in the context of an individual class. However, it is possible to use the E-Portfolio to assess a body of student work over multiple terms. Political Science, for example, has recently agreed to create POSC 400: Portfolio and Presentation. This one-credit course is required in the major. In order to pass the course, students will have to create an E-Portfolio that explicitly articulates how the cumulative body of their political science work meets our departmental learning objectives. The culmination of this work is an oral defense. We are hoping that we can record these defenses and then have the students embed them into their E-Portfolio (and this might be useful for a range of oral presentations). We plan on introducing these requirements next fall and our plan is that students will be working on their E-Portfolio as their work towards completing the major. In doing so, we hope that students will engage in more reflection and metacognition.

Be Sure You Pack Your Librarian

Nearly two years ago on campus, Dr. Joellen Maples waved in my direction, “Get ready!” she laughed. “I am taking my capstone course online… and I am taking my librarian with me!” While I had no idea of the details of her statement, I was thrilled to be included. Joellen’s remark acknowledged the value of Lavery and research skills as a vital part of the course experience. So, let the adventure begin.

Piece of cake, right? Hundreds of times I had conducted 55-minute face-to-face library sessions that positioned students to perform research efficiently and effectively. What could be so difficult about using yet-to-be-determined technologies to transform that same 55-minute session into a virtual experience that would occur without a physical classroom and within a flexible timeframe that allowed students to work at their own pace? Looking back, without a doubt I have had a marvelous growth experience and a ton of fun partnering with Joellen and the College’s Educational Technologist, Katie Sabourin to create an online Library Module tailored to a specific course.

Between Katie and Joellen, I had two capable folks to help me. And, I knew my material well. Still, I had no idea where to start. So as I do when planning a face-to-face session, for my first step I met with the professor so that I could understand my role in the course. Dr. Maples was ready for me. She explained that Week 2 would be where the students would complete the Library Module. The timing was selected so that students would have time beforehand to determine the focus of their literature review. Week 2 signaled the beginning of their literature review research. I would be entering the online dynamic at the optimum moment that librarians call “the point of need”. Student motivation and engagement are at their peak at the moment students are poised to begin research.

My second step was to complete the College’s Online Education Workshop and Fundamentals of Online Teaching, both offered by Katie Sabourin. These are definite Must-Do’s. The structure I learned gave me the theoretical framework for all that I created; it also provided guidance concerning my online interaction with students in the course.

Next, I identified technologies I’d use. For a few weeks I immersed myself in TechSmith’s Camtasia. I broke apart the face-to-face session’s material into 11 segments and wrote scripts explaining each segment. Then, using my scripts and the library’s databases, I created 11 brief online tutorials totaling 26 minutes viewing time. In this way students would be able to view and review each video as needed. I also used Camtasia to create and record a step-by-step overview of the components of the Library Module. All of this material I uploaded to YouTube and then into the Library Module folder in Blackboard. Here is one of the mini-tutorials I created:

Using Doodle, I created a calendar where each student could schedule his or her follow-up one-on-one virtual meeting with me via Blackboard Collaborate. I posted the Doodle link in Blackboard with instructions to the students. I prepared for each virtual meeting by reading the course discussion boards in Blackboard to gain an understanding of each student’s area of research so that when we met online I was able to efficiently discuss their research needs. During our meeting, Collaborate’s screen-sharing feature allowed me to share my screen with a student so that I could model search strategies and database use. For the weeks that followed, I monitored the discussion board in Blackboard in order to make research suggestions if needed.

Dr. Maples selected the Library Module as one of the course’s “accountability moments”. As a result, she requested I create a quiz in Blackboard. Students were required to complete the library quiz with 80% or better in order to continue in the course. Each student was given two attempts to successfully complete the quiz. I structured the quiz with feedback for each question. In this manner, for any incorrect answer, I was able to indicate to the student the specific video in which the correct answer could be found along with a suggestion to review the video before re-taking the quiz.

Each term since summer 2013, I continue to assist Dr. Maples by providing the Library Module for her capstone literacy course. The course always culminates with an evening of student presentations. Dr. Maples plans for two or three virtual performance rooms in Collaborate. In each room over the course of the evening, students present their capstone research to a virtual audience of peers, family, and friends who are able to attend simply by logging onto a computer anywhere with an internet connection. The evening is taped for later access. I have the honor of hosting one of the rooms. What a joy it is to participate in a moment filled with such student accomplishment.

Joellen and I are thrilled by our observations regarding student engagement in the Library Module. The online library module demands active participation by each and every student in the course. To both of us the quality of research being performed in the literature review appears better since moving to the online format as a result of student engagement in the library module. Occasionally, we are even amazed and amused by the unprecedented student embrace of library resources. For example, a student once exclaimed, “Oh, I just love Ulrich’s! It is such a lifesaver!” Haha, rarely does a non-librarian articulate such passion for a library resource.

At the end of each semester, we confirm with each other what we observe individually. When we are engaged, students are engaged; they participate, they support each other, and they enjoy the online experience. In the future, I hope to work with additional faculty to provide a virtual library module as part of their online course. Is every course meant for the online format? Probably not. Still, if you have thoughts of taking your course online, take your librarian with you! The course experience will be the richer for it.

Reflections on Flipping: Three Semesters Later

At some point, I heard about flipping. I heard students liked it but more importantly, I heard it “worked,” anecdotally. As an economist, I wanted to know if it actually worked. Was there a significant improvement in student learning? Could I quantify an effect of flipping the classroom on undergraduates’ learning? My curiosity as an economist and my desire to change up my class, to make it more hands-on and more student-centered, drove me to flip my course.

For a few semesters, I had felt guilty lecturing for an entire class period. I wanted to make class more engaging for the students, who at times looked like they were being tortured. I know, you’re thinking, “It’s econ! How could students not be engaged? How could they not be at the edge of their seats every minute of class time?” Trust me, I hear you. At the same time, I liked to give students hands-on time with the material. I wanted to give them more hands on time with the material. I had heard about some “fun” activities to do in class to illustrate economic concepts, but I had no idea how to add in these activities without cutting material. Of course, I could not cut material. That would be blasphemous. Therefore, flipping, if effective, seemed like the answer to all of my problems.

I also thought if I was going to do this, I was going all in and I was going to flip my entire course. Not only that, but I was going to attempt to quantify the “flipping effect” and determine if there was a significant difference in student outcomes by teaching one class in a traditional manner and one class in a flipped manner. This would enable me to change up my teaching, to do something innovative and breathe new life into my course, and to do some research on the topic.

I next had to decide how I was going to have the students experience the lecture in the flipped class. It was very important to me that the students had some sense of continuity throughout the course, especially as this class structure would be completely foreign for most of them. I also wanted the students to know that I was in it with them, so I decided to make all of my own videos. Furthermore, because I was implementing a treatment-control experimental design, I needed the lecture notes to be the same for the two groups of students.

My basic inspiration for the videos was the Khan Academy. I had seen a couple of those videos, and I really liked how they conveyed information. I believe in economics it is important to see equations solved and graphs drawn by hand, so I knew I wanted the ability to talk through the graphs and equations while writing them out by hand, just as I would do in a typical lecture. Katie Sabourin helped me identify the technology needed to make the videos: Echo 360 to record my computer desktop and voiceover and a Wacom Tablet (pen-tablet) so I could draw on the screen.

All of my videos had the same format: a PowerPoint shell with a black background. I pre-loaded definitions, data, and some text into the PowerPoint presentation, where they would animate upon a click. Using the screen-capture technology, I recorded a voice-over of the PowerPoint slide show, and where appropriate, I annotated the slides using the pen-tablet technology. For example, axes and titles of graphs were animated to “draw” on the screen. Meanwhile, I hand drew the supply and demand curves, shaded in the areas of consumer and producer surplus, and solved calculations by hand. As another example, table shells were pre-loaded into the PowerPoint, but I explained how to fill them out and filled them out by hand on the recordings as I talked through the tables (just as I would do in a lecture class). To keep the videos “interesting” (as if the content weren’t enough!), I included relevant graphics. The videos ranged in length from 4-21 minutes, with an average of 11 minutes.

Here is a short clip from one of my video lectures to give you an example of how the lectures looked online and what I meant by “pre-loaded” and being able to annotate them.  It also illustrates that both the audio and video don’t have to be perfect (which was hard for me to accept as I made the videos):

Interestingly, figuring out how to get the content or lecture to the students was the easiest part. The next step was figuring out what to do during the class time. I now had 3 hours of empty space to fill each week! We started off each class with an open notebook, 5-minute quiz based off of the lecture material from the videos that were due for that class period. This had the additional benefit of incentivizing students to watch, take notes on, and pay attention to the videos. After the quizzes, students engaged in different activities depending on the day and material. They participated in economic experiments, discussed and analyzed popular press news articles or video clips from tv shows, created mind and concept maps, and completed worksheets for each of the activities. Students also completed worksheets identical to those the traditional class completed for each chapter. Finally, they also spent one day per week working on online problem sets. To keep it interesting, sometimes we worked together as a class as a whole, while other times students worked in small groups, pairs, or individually, depending on the content and the activity. While the students were engaged with the activities, I circled the room and answered questions that they had. Actually, I tried not to answer the questions that students had, but instead, I tried to get them to learn how to figure out the answers for themselves. Sometimes, if I noticed the same question over and over, I’d have them take a mini-break while I did a mini-lecture.

This is my third semester teaching in a flipped format. Why do I continue to do it? First, it works. My analysis indicated that students in the flipped class scored significantly higher than in the traditional class on midterm and final exams. Controlling for student academic and demographic characteristics, the effect had a lower bound of roughly two-thirds to an entire letter grade. (Note: It worked in economics. It worked in a small, introductory economics course. My findings do not indicate that it will work in every single class. In fact, I have no doubt that there are classes out there for which this structure just wouldn’t work. I also can think of other classes of mine for which a full flip wouldn’t work either, but a partial flip might. So, it still depends on the instructor’s desire to flip, the content and course, and a host of other things.) Second, the students seemed to like it. Okay, they didn’t hate it. Some of them really liked it. Some of them indicated that this structure taught them about themselves as learners! Once I saw that flipping worked, and once I saw that the majority of students didn’t hate it, I wasn’t sure how I could go back to teaching this class in a traditional manner.

From a personal standpoint, the marginal cost of flipping from one semester to the next is pretty small. I also really enjoy the format, as each day is a little different. Every semester, I look to improve the flip, change activities, make sure activities work out the way they should, etc. I still think that there are some topics that are better suited to a flipped structure than others. I continually worry that class is not engaging enough, or that students resent having so much problem solving and group work in class when they might want lecture (okay, maybe I’m projecting here, because I loved lectures and was not a fan of group work as a student). I’ve found that flipping helps me to get to know my students more. I have a better sense of who is keeping up with the material and who is falling behind.

Flipping has also changed my outlook on some of the other classes I teach. When you flip your class, you have to change the way you think about it. You start to question things you teach, why you teach them, and how you teach them. I find this is helping me in my other classes as well, and I am working to identify different ways to keep improving my courses. For example, in the future, I want to implement partial flips in different classes, such as statistics and econometrics. I think there are some topics that are appropriate for out of class delivery and some topics for which I really must do a “live” lecture.

My advice: flip a class you know. Flip a class where you can anticipate the questions because you already know where the students struggle. It will also help you to decide how to best utilize your class time.

 

Digital Badges: What Are They & What Can We Do With Them?

Have you heard of badges? No, I don’t mean those that are worn by police or military officials, or even merit badges collected by boy/girl scouts and displayed so proudly on the child’s uniform, though digital badges are somewhat modeled after these badging systems. According to the MacArthur Foundation, “a digital badge is a validated indicator of accomplishment, skill, quality, or interest that can be earned in many learning environments.” Similar to the physical badges that signify the completion of a task or acquired skill level, digital badges can be used to visually display a wide variety of skills and competencies online, including both physical and virtual skills, as well as hard and soft skills. Digital badges also take some principles from video game design as they can be used as a reward for completion of a task or a means to unlock additional tasks that must be completed in a sequential order.

So, now that we know a little more about what digital badges are, why would we want to use them in an educational setting? In general, digital badges are a relatively new educational technology so the answer to this question is evolving. However, there is a significant amount of research on topics that overlap with the use of digital badges like the effects of student motivation, both intrinsic and extrinsic, on achievement, as well as the use of rewards and recognition within virtual communities. These techniques can be implemented in a variety of ways, but digital badges have become a new option to incorporate these strategies into educational settings. Digital badges really started to get attention in 2011 when the Mozilla Foundation announced the Open Badges Project, which provides a set of standards for creating, credentialing and displaying badges of all kinds. Without this type of underlying infrastructure for badges there would not be the attention from the education market that we see today. Digital badges also overlap with ePortfolio technology, as one of the main components of collecting badges is to display them to others as evidence of your skills or accomplishments on given topics. ePortfolios can be a useful place to display this type of information for academic and professional reasons. Digital badges have also generated discussion around the tracking and credentialing of informal learning. With the increase in MOOCs and other non-credit educational experiences and the issue of how to assess and award credit for these types of learning experiences, the use of digital badges have been suggested as one of many possible solutions.

Please take a few minutes to watch the following video to learn more about digital badges and how they may be used within education.

Source: MacArthur Foundation (http://www.hastac.org/digital-badges)

Below are a few examples of those using digital badges in higher education:

University of California, Davis – Sustainable Agriculture and Food Systems Program

“Instead of being built around major requirements and grades in standard three-credit courses, the Davis badge system is based on the sustainable-agriculture program’s core competencies—”systems thinking,” for example. It is designed to organize evidence of both formal and informal learning, from within traditional higher education and without.”

Read the full article here: http://chronicle.com/article/A-Future-Full-of-Badges/131455/

Purdue’s Passport Project

“In one early example of Passport’s use, instructors are giving out badges for students who pass an 8-week MOOC-like course in nanotechnology that doesn’t have credit attached. In another example, the provost’s office has created a badge related to intercultural learning that students can earn for their work in different disciplines and departments.”

Read the full article here: http://campustechnology.com/Articles/2013/06/20/How-Badges-Really-Work-in-Higher-Education.aspx?Page=3

Carnegie Mellon University – Computer Science Program

“Carnegie Mellon University has included badges in the Computer Science Student Network (CS2N), which provides a distributed learning infrastructure for computer science and STEM skills. Learners participate in a scaffold environment and work on achievements ranging from entry-level skills to industry certification. Badge pathways provide a clear view of progress, as learners can clearly see how lower-level competencies lead to higher-level competencies. Creative competitions provide additional motivations and opportunities for peer review and learning from others’ work. Learners can progress to the levels of achievement that tie into industry-accepted certifications and entries to employment.”

Read the full article here: http://edtechdigest.wordpress.com/2013/06/28/put-a-badge-on-it/

As well as the above examples, Blackboard has also recently released a tool to allow faculty to create badges and award them to students directly within their courses. The Blackboard Achievements tool connects with the Mozilla Open Backpack project, so that students can display badges acquired within a course on a public page for individual use. Here at Fisher, the Achievements tool was part of the recent upgrade to Blackboard that was completed over the holiday break. If you would like to know more about the Achievements tool, please contact me directly and we can discuss your own badging ideas together.

Would you like to learn more about digital badges and discuss their use in education with other interested faculty at Fisher? On behalf of the Educational Technology Roundtable and the Fisher Geeks Blog, we would like to invite all interested faculty to participate in the following webinar sponsored by Pearson and hosted by The Chronicle of Higher Education, as well as participate in an open discussion following the webinar.

Designing Your Own Open Digital Badge Ecosystem

Thursday, February 6, 2014
 2:00 p.m. in Nursing 209

For a full description of the webinar, please click here.

The Power of Learning Analytics: The Wegmans School of Pharmacy Approach

As a follow-up post to The Power of Learning Analytics from last spring, guest blogger Jane M. Souza, Ph.D., Assistant Dean of Assessment in the Wegmans School of Pharmacy at St. John Fisher College, shares the strategies used within the school to collect, analyze and continuously improve the Pharmacy program through the use of the computer-based exam management software, ExamSoft and the School of Pharmacy assessment process.

All courses have learning outcomes listed on the syllabi. All faculty members strive to teach learning outcomes in ways that encourage student achievement. These facts are nothing new and are common among any undergraduate or graduate level course, but the way faculty in the School of Pharmacy are looking at student progress on learning outcomes is new. Rather than relying solely on test grades to assess student progress, they are tracking density of coverage on each course outcome along with whole-class as well as individual student achievement within each of those areas. How are they managing that much data? They are testing in an electronic environment using ExamSoft. The implications for change management at the course level are outstanding and only exceeded by the ability to drive quality improvement at the program level and opportunities for self-monitoring at the student level.

So how does this work exactly? It all begins with pulling data out of existing exams maximizing the potential of embedded assessments. Test items, or questions, have been crafted and honed by faculty over years of teaching experience. The items include true/false, fill-in-the-blank, multiple choice and essay questions. Using ExamSoft, each item is electronically tagged or coded to identify the question on multiple levels including the following: learning outcome, program standard, accreditation standard, and level in Bloom’s Taxonomy (e.g. knowledge, application, synthesis). Questions can be tagged several times as needed. For example, one question may be addressing course learning outcomes 1, 3; program outcomes 1e, 2g. 3a; accreditation standards, 16.2, 32.1; and application level.

When students take their exams, data is collected for each test item. Faculty are provided a wealth of information previously unavailable to them. They know how much time, to the second, students spend on each question – both individually and as a class.  WSOP_1This is great information for time management and test construction. They know how many questions they asked per exam and per semester on each learning outcome. Most importantly, they know the student achievement on each of these learning outcomes. Consider the two tables to the right and below. The table to the right shows the course-level information available prior to electronic testing. The table below shows the detail currently available and how faculty might use the data.

WSOP_2

From the tables above, it is easy to see how faculty can make evidence-based changes to their classes. The same information can drive curricular reform. Consider the old-style curriculum map. It documents that the curriculum has been taught, but does not record what was learned.

Old Style Curriculum Map

WSOP_3

Evidence-Based Curriculum Map with Density of Coverage and Student Performance (note the opportunity for improvement)

WSOP_4

At the student level, individualized learning analytics help students consider their own strengths and opportunities for improvement. Whereas, in the past students were monitoring their progress through course grades, they are now able to reflect on their strengths and weakness at the learning outcomes level. Previously hidden deficiencies are now revealed and can be addressed promptly.WSOP_5

The student represented in the table to the right appears to be strong in each area, but are there hidden deficiencies?

The student may have a lack of understanding for a single learning outcome, but the deficiency is masked by high achievement in other areas. When students are able to see their performance at the learning outcome level, as shown below, they are able to work on remediation well before they encounter critical assessments such as exit or licensing exams.

WSOP_6

The learning analytics made possible through electronic testing have been put to good use in the Wegmans School of Pharmacy. The faculty are employing the data to make course-level changes, the students are addressing their areas of weakness, and curriculum and assessment committees are analyzing the data for continuous improvement at the program level.

Feel free to visit us at Wegmans School of Pharmacy to see this process in action, or stop by and see me. I am always happy to talk assessment.

Jane M. Souza, Ph.D.
Assistant Dean of Assessment, Wegmans School of Pharmacy

The Power of Learning Analytics

In the digital world we live in today, we all know the endless amount of data that is collected throughout the many interactions that occur in our daily lives. You can see the evidence of this type of data in a variety of ways, including Amazon’s recommendation feature that suggests what products you might find interesting based on your previous purchases and what similar customers also purchased, as well as the Netflix movie recommendations that are created based on your ratings of previous movies you’ve watched. You probably won’t be surprised to know that even one of the large, local companies (where I know I spend too much time and money) uses this technology as well. Wegmans uses the data from Shoppers Club Cards for a variety of purposes, including sending out coupons and making sure the right products are stocked based on store locations. In at least one case, Wegmans even contacted shoppers who purchased a certain brand of soup to let them know about a recall on the product. We all create these kind of data points every day without even noticing it. Companies have found a way to utilize this type of data for marketing, advertising and to create better products and services based on what they have learned about their customers. This strategy of data mining for business purposes has been going on for many years.

With the increased use of technology and web-based tools in education, students have also left data footprints as they travel through their educational activities online. This data often goes untouched by many educators; however, there is a growing trend to make better use of this data to increase student learning. Many educational technology companies are beginning to focus on better utilization of this data and enhanced reporting to allow educators to use the data in more effective ways.

When we hear “Khan Academy”, most people immediately think of a short video with a black background, annotated colorful text and someone talking through a complex math theory. This is a large part of what Khan Academy offers, but a key component of their overall design is the tracking of student progress through a series of these short videos and accompanying activities over time. This data is collected while the student watches videos and completes short practice questions and concept quizzes during and after each skill checkpoint. The data collected includes not only the answer to the quiz questions, but data on how many times a student has viewed a certain video, how long they spent on it, what time of day it was viewed and more. Check out this Khan Academy data reports page for a full list of the reports they offer to educators to track the progress of students.

Knewton is another company innovating new ways to utilize the vast amount of data created by students throughout their learning experiences online to create tailored and more personalized learning environments for each student. Check out the below video from Knewton’s website.

Many publishers, including Pearson, Houghton Mifflin and Wiley, partner with Knewton in order to apply their adaptive learning technology to the large amount of content that publishers have created over the years. The combination of resources, on one side focused on the analytics and the other on high quality content, create innovative, data driven tools that can quickly be used by students and instructors in a variety of course and content areas.

Now that we have discussed the type of data that is tracked and how it has been used in a few examples to create personal learning environments for students and valuable reports for instructors, you might be wondering, “How can I apply this idea to my own courses?”

First of all, both examples listed above, Khan Academy and Knewton, provide content that utilize this type of data collection and reporting to instructors in many disciplines. Check to see what might be available for you. Also, check with your publishers if there are other resources that might give you this type of information. If you are using other technologies in your courses, check to see what type of statistics the tool is tracking already on student usage and if the reports of this data might be useful to you.

If you use Blackboard in any way, even just to post a syllabus document, there is already data available at your fingertips. You can quickly see the last time a student has logged into your course, as well as if they clicked on the file that you uploaded for them to read before the next class. The amount of data available for you to view in reports increases with your use of the variety of Blackboard tools. For example, you can view how many times a user has clicked on a specific content area, as well as other tools like activity in discussion forums and groups. These reports provide the specifics on each student, as well as the common days and times that content is most used in your course overall. Combine this data with the results of a short quiz activity and you would be able to acquire similar data that is made available to users of Khan Academy resources.

Blackboard also provides tools for early warning messages based on predetermined rules. If a student meets a certain criteria identified by the instructor as “at risk” on a certain assignment or overall in the course, the instructor is notified and is prompted to send a notification to the student directly. Blackboard also provides adaptive release tools that allow instructors to control the release of certain course material based on the successful completion of previous items.

A variety of other commonly used tools at Fisher provide similar types of data and reporting features to instructors. Echo360, for example provides instructors with a easy to view report that includes data on how many times a video recording has been viewed by unique viewers and overall, as well as a heat map of the video indicating points that have been viewed most frequently by all students. These points may indicate areas of confusion where content was viewed multiple times. These statistics can be very handy if you are teaching with the flipped classroom model, which we discussed in a previous post, in order to know who is viewing each recording prior to class and to identify areas that might need further clarification in the next face-to-face class session with the larger group. Below shows an example of the Echo360 statistics.

Echo360 stats

The data provided by a variety of educational tools can be used by instructors to paint a more accurate picture of where students are spending time, where they may be struggling with content and students who may not be fully engaged with the course. The ability to access and utilize this data can help all instructors make more informed decisions in communications with individual students, as well as for course development and planning overall.