Biomarkers of Motor and Cognitive Function (MovAlyzeR use starts at Minute 3:57). Jihye Ryu, Joe Vero, Roseanne D. Dobkin, Elizabeth B. Torres (2019). Dynamic Digital Biomarkers of Motor and Cognitive Function in Parkinson's Disease. Video [10:28] JoVE, issue 149, e59827, doi:10.3791/59827. https://www.jove.com/v/59827/dynamic-digital-biomarkers-motor-cognitive-function-parkinson-s 

The MovAlyzeR test included (2. Experimental Procedure):
Tasks 1, 7: Drawing the Benson Complex Figure, Immediate and Delayed (4:14)
Tasks 2, 3, 4, 5: Performing the Trail Making Tests A and B (5:04),
Task 6: Performing the Clock-Drawing Test (5:28).

Handwriting Analysis Indicates Dyskinesias (MovAlyzeR use starts at Minute 0:24). Dean, D. J., Teulings, H. L., Caligiuri, M., Mittal, V. A. (2013). Handwriting Analysis Indicates Spontaneous Dyskinesias in Neuroleptic Naïve Adolescents at High Risk for Psychosis. Video [5:52] J. Vis. Exp.(Jove) (81), e50852, doi:10.3791/50852 (2013). https://www.jove.com/t/50852/handwriting-analysis-indicates-spontaneous-dyskinesias-neuroleptic 

Import example experiment into your own User ID before editing!
Step 1. Select the Example User UU1 (File >Users)
Step 2. Export an entire experiment to a .MEF fie (Right-click on it >Export)
Step 3. Create or select your own user (File >Users)
Step 4. Import the the .MEF file of the experiment (File >Import)

Comparing left and right hands (Experiment E01). A simple MS Windows PC, a pen tablet with USB connection, and the MovAlyzeR MovAlyzeR software forms one of the most accurate handwriting and human movement control measurement systems available on the market. As an example, we compare right and left hand performance to examine the extent of motor equivalence. The handwriting patterns are segmented into up and down strokes. For the first 8 strokes we show per stroke:

o Duration            (in seconds)
o Absolute size       (in centimeters) 
o Normalized jerk     (unitless) (i.e., dysfluency)
o Axial pen pressure  (in device units)

Vertical bars are either + or 1 one standard deviation (SD) or standard error (SE, i.e., standard deviation of the average):

SE = SD / sqrt(N-1) 

The SE enables us to quickly estimate which differences could be significant:

 32% of the data will be between + and -1*SE,
  5% of the data will be between + and -2*SE, 
0.3% of the data will be between + and -3*SE. 

So if double the SE bars of two points are not overlapping these two points will be likely significantly different (p<5%).

Recording a paragraph and segmenting into words (Experiment WRD). MovAlyzeR's recording time has been artificially reduced till 1800 seconds (i.e., 30 minutes). MovAlyzeR has an option of Word Extraction:. A "word" can actually be a line, word, or pen down segment depending upon the settings in:

Right-click your experiment >Processing >Advanced >Word Extraction

Caligiuri's extrapyramidal side effect test, Phase 2 (Experiment PH2). (no sound). Record various simple and more complex handwriting patterns. The patterns needs written between 2 lines at a distance of 1 cm (TOP zone), 2 cm (MID zone), and 4 cm (BOT zone). See Caligiuri et al., (2006). https://pubmed.ncbi.nlm.nih.gov/19692133/?from_single_result=caligiuri+eps+movalyzer

Tremor (Experiment TRE) (no sound). The test consists of recording the pen tip while producing maximum-speed circles and by holding the pen tip still above the paper. The circles yield a sharp spectrum (in this writer 2.2 Hz -- the maximum humans can produce circles is 5 Hz). The horizontal axis is logarithmic to visualize a large frequency range. The blue curve is the low-pass filtered curve. The red curve is the unfiltered movement.

Simulated data generator (Experiment EX9) (no sound). Back-and-forth strokes, ellipses, i-loop patterns can be generated using known features, so that you can verify under which conditions MovAlyzeR will estimate the correct features. MovAlyzeR as been developed to produce unbiased features. Features that can be set include:

o Start time, 
o Stroke duration, 
o Number of strokes, 
o Trail time (i.e., time of rest after completing all strokes),
o Vertical stroke size, 
o Duration of primary submovements,
o Random noise, 
o Quantization noise

Choice reaction time -- Hick's law (Experiment EX1). (no sound). Reaction time (RT) increases logarithmically with the number of choices. Movement time (MT) is not increasing with the number of choices. MT increases mainly with the required precision of the movement to the target. Various factors will affect RT, e.g., advance knowledge (via the Precue stimulus), arrangement, order, familiarity, ... Two trials are shown. The first one requires an 8-choice. The second one requires a 4-choice. Recording is set to start with the imperative stimulus. RT is the StartTime of Stroke 1. MT is the Duration of Stroke 1. After each trial the participant receives feedback (Knowledge of Results -- KR) of two movement features: RT (Feedback-1) and MT (Feedback-2) in seconds. KR motivates participants.

Menu layout: 16 dropdown items in one row vs. in 4 columns of 4 rows (Experiment LAY) (no sound). See https://www.dropbox.com/s/8vcs2drcby3dpu0/Teulings_Gemmert_IGS2003.pdf?dl=0

Movement time test -- Fitts' law (Experiment EX3). (no sound) Movement time (MT) increases with the required precision of back and forth movements between two targets. Precision requirement can be increased by reducing target size.

Fitts' law test (Experiment EX3) See above. Here sound is added to the event of "Ready to begin recording". The voice is generated by text-to-voice systems, producing a .wav sound file. After each trial Knowledge of Results of reaction time (RT) and movement time (MT) is added

Mirror writing in real life (Experiment IMG). Tracing of a star while visual feedback is mirrored is a challenging visuo-motor learning task. Mirror writing is not as simple as it looks.

Mirror writing (Experiment IMG). The visual feedback during writing can be manipulated by specifying horizontal and the vertical gain factors and rotation angles. The transformation where the vertical dimension is expanded while writing on a display tablet causes micrographia in Parkinson's disease patients to show macrographia. E.g., you can:

o Mirror (e.g., X-gain = -1)
o Expand vertical size but not horizontal size (e.g., Y-gain = 1.3) 
o Slant  (e.g., y-rotation = 30 degrees)
o Rotate (e.g., X-rotation = 30 degrees, Y-rotation = 30 degrees)

The sound at 0:41 like a stone fell in the water is an option in the experiment properties to alert the user that samples were missing. This cn happen when the pen is lifted beyond the proximity distance (e.g., 1 cm) or when the pen has been moved outside the recording area of the tablet.

Right-click your experiment >Properties >Advanced >Discontinuity >Check: Beep at each data discontinuity?

Learning of handwriting (Experiment LEA) (no sound). Submovement analysis was applied to compare learning cursive handwriting versus block print. See https://www.dropbox.com/s/n2u91gteg4fo69x/Teulings_Romero_IGS2003.pdf?dl=0

Teacher writes the words (Experiment TEA) then the student monitors the movement and tries to perform these words (Experiment STU) (no sound). The teacher performs several words. The words will be immediately available to the student. The precue stimulus shows how the word is performed by a red dot traveling at real speed over the blue ink. Then the student will be able to perform the word. The student's fluency is presented after each word (e.g., Average Normalized Jerk should be 7 - 150).

Teacher writes the words (Experiment TEA) then the student monitors the movement and tries to perform these words (Experiment STU) (no sound) Similar as above but now the teacher writes examples on a blackboard. The students are presented with the writing pattern and a small dot traveling over the ink.

Write with pen, mouse and with finger (Experiment TOU) (no sound). For this experiment we use a pen tablet with touch input. The input device is set to mouse.

Employee Status Program (ESP) This 1-minute test can be used as a non-invasive, adulteration-proof, unsupervised, remote test for substance abuse and fatigue using the MovAlyzeRx Graphical User Interface (GUI -- included in the MovAlyzeR suite).

The employee takes a seat, 
logs in, 
and performs several simple handwriting patterns. 

Immediately after the test the results are assessed. When the employee's performance is about 2 standard deviations from his/her past performance the supervisor will receive an alert. See MovAlyzeR's Norm Database system.

Handwriting impairment test (Experiment ALC) (no sound). Same as above. This example test is based on movement amplitude, speed, and fluency and includes multiple-choice questions. This test is adulteration-proof as it measures biometric properties of a person's handwriting, enabling to verify that the intended person is performing the handwriting patterns.

Handwriting impairment test (Experiment ALC) using MovAlyzeRx (Rx) instead of MovAlyzeR (no sound). Same experiment as above but now performed using MovAlyzeRx. It is part of the MovAlyzeR suite.

o For running and analyzing experiments without options to alter tests.
o Simplified interface with only buttons without dropdown menus.
o 6 Windows that can be resized, rearranged, undocked, hidden behind a tab, etc. enabling to create a personal interface 
o Experiment-centered data structure of MovAlyzeR
Experiments                        = i.e., Tests
 +--Groups                         = i.e., Visits
 |    +--Subjects or participants  = i.e., Patient
 |        +--Trials
 |
 +--Condition                      = i.e., Task

o Becomes patient-centered with appropriate vocabulary:
Patients 
 +--Tests (per Patient) 
     +--Task (per Test) 
     |   +--Trials (per Task)
     |       +--Graphs and data (per Trial) 
     |
     +--Visits (per Test and per Patient)

The test procedure is:
1. Login
2. Type patient code.  
      o The lists of patient, tests, visits, Tasks, Trials reduce incrementally. 
      o If no patient is found the patient code will be entered as a new code.
3. Run test 
      o by default the most recent test for this patient will be run.
      o By default the next Visit number after the last Visit number will be created.
4. Logout

Line thickness varies with pen pressure (Experiment BPP). (Sound is from the computer speaker). Line thickness varies during writing with axial pen pressure. If pressure exceeds a settable value an ugly beep is produced. The yellow notepad paper is a stimulus that you can add per condition.

Beep at high pen pressure (Experiment BPP). (Sound is from the computer speaker). Similar as above, but, in addition, feedback (Knowledge of Results or KR) of the average pen pressure is presented graphically and numerically after each trial.

Beep at high pen pressure (Experiment BPP) (Sound is from the computer speaker). Similar as above, but in addition, after atrial the processed data can be shown by double-clicking on the completed trial. The writing movement is segmented into up and down strokes (red circles). The axial pen pressure is fed back by the line color(light red is low, dark blue is high). Click in the chart and use the right and left keyboard arrows to move the crosshair along the handwriting trace. Above the chart are shown for each movement in time:

o Sample number, 
o Stroke number, 
o Segment number (If Submovement Analysis not checked, this will equal stroke number), 
o Axial pen pressure (in device units),
o Normalized jerk / Stroke (i.e., dysfluency) (unitless), 
o Vertical stroke size (in cm), 
o Surface of the loop formed by the current stroke and the previous stroke (in cm2). 
The extracted data file will specify more decimals. 

Calling Matlab scripts from MovAlyzeR (Experiment MAT) (callouts but no sound).

Bimanual coordination (Experiment 2PN) (no sound). Wacom Intuos, and Intuos2 pen tablets enables recording 2 pens. For additional info, please, visit, https://neuroscript.net/forum/showthread.php?2894-2PN-Two-Pen-Experiment.

Processing handwriting images (Experiment SFX) (no sound).

Processing spirals written on paper (no sound).

MovAlyzeR Overview. Getting started page, Tree view pane, Instruction pane, Recording pane, Output pane, Example user UU1, Experiments, Groups, Subjects (i.e., Participants), Conditions (i.e., Tasks), Experiment settings, Condition experiments, Stimuli, Red on yellow settings are non-default, Consistency checking, Number of trials per condition, Device-setup wizard, Run-experiment wizard, Subject instructions, Summarize, Analysis chart, Reports.

Download and Install MovAlyzeR

Steps:
 1. Google: Handwriting movement software
 2. Visit www.neuroscript.net >Products >Download
 3. Sign-up
 4. Receive Download instructions by email
 5. Download and save MovAlyzeR*.msi
 6. Open MovAlyzeR*.msi
 7. Run MovAlyzeR.exe
 8. Receive 16-character Request Code from your computer
 9. Email this code to sales@neuroscriptsoftware.com (Do not use online activation)
10. Receive Trial Activation Code within 1 business day
11. Enter Trial Activation Code
12. Email sales@neuroscriptsoftware.com for additional Trial Activation Codes for the same computer.

Pen tablet driver download and install. MovAlyzeR will record the pen movements of the pen tablet while the pen tablet is still working as a mouse. To be able to record the pen at a fixed, high sampling rate and to be able to record pen pressure, the WinTab compliant tablet driver needs to be installed. You can verify that the tablet driver has been installed if you can select tablet as device. Furthermore,, each user needs to run the input device setup wizard to ensure that MovAlyzeR knows about resolution, sampling rate and maximum pen pressure.

Pen tablet device setup.

Start MovAlyzeR (callouts but no sound).

Record handwriting with MovAlyzeR

Create new user. When sharing MovAlyzeR export files (.MEF) in the future it will be important to specify also your Site ID and Site Description. Furthermore, in multi-site studies it may be important to set Subject ID Ranges by selecting non-overlapping Start and End IDs to prevent clashes of Subject IDs, or you can uncheck: "MovAlyzeR generates subject IDs automatically?"

Experiment setup.

Experiment data structure. (callouts but no sound).

Knowledge of Results (KR). (callouts but no sound).

Stimulus element duplication (no sound). Each condition can have audible and visual stimuli. There are 3 visual stimuli of settable durations and latencies: Warning Precue, Imperative stimulus. The imperative stimulus is the start of the recording. A visual stimuli consist of stimulus elements. They have many settings. Therefore it can be useful to duplicate a stimulus element and change just one setting while all other settings will remain unchanged.

Experiment
 +-Condition1
 |  +-Audible Stimulus
 |  |
 |  +-Visual Stimuli 
 |     +-Warning Stimulus
 |     |  +-Element1
 |     |  +-Element2
 |     |  +-...
 |     |
 |     +-Precue Stimulus
 |     |  +-Element1
 |     |  +-Element2
 |     |  +-...
 |     |
 |     +-Imperative Stimulus
 |        +-Element1
 |        +-Element2
 |        +-...
 |
 +-Condition2
 |  +-Audible Stimulus

Run Experiment.

Raw pen data and processed data of a trial (voice and callouts). The processed data consist of low-pass filtered positions and its derived time functions.

0th time derivative: Position
    X (i.e., horizontal position), 
    Y (i.e., vertical position), 
    Z (i.e., axial pen pressure), 
1st time derivative: Velocity (i.e., change of position with time)
    Vx 
    Vy
    Vabs (i.e., change of position with time, or absolute velocity)*
2nd time derivative: Acceleration (i.e., change of velocity with time due to force)
    Ax
    Ay
    Aabs 
3rd time derivative: Jerk ( i.e., change of acceleration with time due to change of force)
    Jx
    Jy
    Jabs

Absolute Velocity (Vabs) is the addition of horizontal velocity (Vx) and vertical velocity (Vy). Another name is Tangential velocity, i.e., velocity in the direction of the movement. While Vx and Vy can be positive (i.e., upward or rightward) or negative (i.e., downward or leftward) Vabs is only 0 or positive.

Vabs = Sqrt (Vx^2 + Vy^2)

Chart and view processed trials (callouts but no sound). Expand an experiment into groups and a group into subjects (i.e., participants or writers), and a subject into trials. Double click on a trial, Produce additional charts of the same trial, and choose different Y and X axes and color feedbacks. View also the extracted stroke features per stroke or segment. Click in a chart and use the right and left arrow keys of the keyboard to walk through a recorded trial.

Submovement Analysis (no sound). The participant is instructed to produce back and forth strokes between 2 targets. The targets are stimulus elements of the imperative stimulus. The red circles mark the beginnings the successive strokes. In the Vy vs. Time chart the circles separate the up (positive Vy phases) and down (negative Vy phases) strokes. The first upward stroke shows a lot of hesitations. The first hesitation per stroke is marked by a diagonal cross (x). The lack diagonal cross separate the primary submovement (i.e., the ballistic, preprogrammed part of the stroke) and the secondary submovement (i.e., the part where pen gradually homes into the target). The smaller the secondary submovement, the more fluent the stroke. By default, submovement analysis is off.

Right-click your experiment >Properties >Processing >Segmentation: 
  Check: Submovement analysis
Right-click your experiment >Process >Reprocess All Trials

Consistency Check (no sound). Immediately after a trial is recorded the consistency of the trial with the condition requirements are checked. For example, if the trial requires a complex loop pattern of a particular size but the participant produces a different loop pattern, the trial will be marked t with a green check mark but with a red check mark. Optionally, the the experiment will be interrupted so that the experimenter can decide to redo or accept the trial. This will help to collect the required trials. to facilitate automatic processing we want the intended conditions in each trial. Ideally we want to exclude human decisions or touch-ups during data processing so that data collection and processing are as close as possible to double blind.

Summarize Tutorial.

Summarize and Analyze Experiment (callouts but no sound).

Experiment Reports (no sound). Reports can be produced of all experimental settings, or all conditions, or all participants (subjects) with when and how many consistent trials were produced and unusual tablet settings.

Norm Database. Building simple pre-test / post-test experiment and verifying whether a participant (i.e., subject) is significantly different from his/her previous test or whether the participant's data are within the population of all participants.

Norm Database to verify a new participant fits within the tested population.

Norm Database to compare person with his/her previous performance and the average performance

Forensic application. This is a demo showing how Movalyzer processes online and offline handwriting.
The online handwriting movements that were recorded using a pen tablet at 5080 lines per inch. These data enable us to estimate dynamic features and shape features. Pen pressure can be represented by different colors. The segmentation was based on the vertical velocity zero crossings, i.e., into up and down strokes. Other segmentation methods are available.
The offline handwriting traces on paper that were scanned using an optical image scanner at 600 lines per inch. The images enable us only to estimate shape features. Dynamic features such as direction, velocity, and pen pressure cannot be estimated. Line thickness can be resprented by different colors. The segmentation was based on stroke end points and stroke fork points. Very short segments that have no relevant shape features and long horizontal segments that are likely rule lines are omitted.

PDF by Hand -- Annotating PDF files by hand. PDF by Hand is a separate software package enabling you to draw on or sign PDF files using your pen tablet.

Unwritten (2008) [4:45]. We Live; We Work; We Think; We Create; We Play; and We Write but the rest is ... still unwritten ...

Produced by Barry Yancosek and MAJ Katie Yancosek, MS, OTR/L, CHT. (September, 2008).

Unwritten©,. Lyrics by Natasha Bedingfield, Danielle Brisebois, & Wayne Rodrigues. Sung by Natasha Bedingfield.