ABSTRACT
Kinematic measurements have been essential to the study of comparative biomechanics and offer insight into relationships between technological development and scientific progress. Here, we review the 100 year history of kinematic measurements in Journal of Experimental Biology (JEB) through eras that used film, analog video and digital video, and approaches that have circumvented the use of image capture. This history originated with the career of Sir James Gray and has since evolved over the generations of investigators that have followed. Although some JEB studies have featured technological developments that were ahead of their time, the vast majority of research adopted equipment that was broadly available through the consumer or industrial markets. We found that across eras, an emphasis on high-speed phenomena outpaced the growth of the number of articles published by JEB and the size of datasets increased significantly. Despite these advances, the number of species studied within individual reports has not differed significantly over time. Therefore, we find that advances in technology have helped to enable a growth in the number of JEB studies that have included kinematic measurements, contributed to an emphasis on high-speed phenomena, and yielded biomechanical studies that are more data rich, but are no more comparative now than in previous decades.
Introduction
Measurements of animal motion are fundamental to the study of comparative biomechanics. This field originated largely through the development of kinematic techniques and the mathematical modeling of forces from these measurements. This approach was conceived to a great extent through the published research of the first editor of Journal of Experimental Biology (JEB), Sir James Gray, as chronicled in a series of fascinating ‘JEB Classics’ articles (Lauder and Tytell, 2004; Fish, 2005; Bertram, 2007; Brokaw, 2006). Gray, along with numerous colleagues from Cambridge University, developed a methodology for the study of animal locomotion that bears a strong resemblance to present-day research on this topic. However, the march of technological progress and advances in scientific practices have since shaped the evolution of research in comparative biomechanics. The primary aim of the present Review is to trace this evolution by focusing on how kinematic measurements have changed and how those alterations have influenced the nature of articles published in JEB.
A reading of JEB's volumes reveals three major eras in the history of kinematic recordings. The film era marks the advent of comparative biomechanics, which was enabled by the ubiquity of film cameras for still photography and cinematographic (i.e. ciné or movie) recordings (Fig. 1). The analog video era ushered in a period when long-duration recordings could easily be reviewed, at an inferior resolution to ciné recordings. The digital video era enabled investigators to capitalize on innovations in consumer electronics and computing to record and analyze video entirely with digital technology in the new millennium. Contemporary digital recordings now exceed film resolution and have opened the door for automation to ease the acquisition of kinematics from images. Throughout this history, investigators have sought ways of acquiring kinematics that circumvent the demands of recording, storing and analyzing photographs and movies. Our consideration of changes in kinematic measurements is organized around these major eras of image-based kinematics and the techniques that have avoided the use of image capture, which entails the the storage of movies.
The film era (from the 1920s to the 1980s)
Technological origins
Kinematic measurements in the film era are characterized by the use of photographic film stock for either still photography or ciné recordings. This film consisted of flexible cellulose acetate or nitrocellulose strips, 35, 16 or 8 mm wide, with a series of rectangular perforations lining the edge of one or both sides. When manufactured, a gelatin emulsion with light-sensitive silver halide crystals was layered upon the film before being rolled up into an opaque package in the dark. This film was loaded in a camera that included the optics and aperture to focus and control the diameter of light from a subject, a mechanical shutter to control the exposure duration, and an intermittent mechanism that engaged the perforations with claws to move the film into the light's path and to keep it still during an exposure. By the time they were used in JEB articles, movie cameras included an electric motor that would drive the shutter and intermittent mechanism together so that the film moved only between exposures. Depending on the film type, its chemical developing yielded either a positive image for the projection of movies or a negative image for generating photographs (Lipton, 2021).
The power of movies and still photography for kinematics was demonstrated before JEB's first issue, in the late 19th century, through the efforts of Étienne-Jules Marey. James Gray later credited Marey, along with the 17th century physiologist Giovanni Borelli, as offering the most important contributions for the study of animal locomotion since Leonardo da Vinci (Gray, 1968). Marey was a physician, inventor and scientist who was driven through much of his adult life by an ambition to reveal the mechanisms of locomotion. Inspired by the early chronophotography (i.e. a series of still photos of motion) of Eadweard Muybridge, Marey shifted his attention away from the design of instruments for physiological diagnostics and mechanical measurements of animal motion, toward the building of cameras that permitted kinematic measurements with high precision (Braun, 1992). By conducting research at his own physiological station in the 16th arrondissement of Paris, Marey published regularly in the pages of the weekly reports of the French Academy of Sciences (Académie des Sciences; e.g. Marey, 1888b,a) and a series of books (e.g. Marey, 1874, 1894).
Marey's techniques included a variety of forms of chronophotography, first using photo-sensitive plates and eventually with film. By the late 1870s, light-sensitive plates and mechanical shutters made it possible to produce an image with a sufficiently brief exposure to capture a fast-moving subject without blur. Muybridge used an array of these cameras to capture a sequence of images of a galloping horse (Solnit, 2004), but the trigger mechanism and distortions introduced by multiple perspectives were insufficiently precise for Marey's scientific standards. To fix the perspective, Marey's early chronographic camera designs included a rotating metal disk in the light path with perforations that allowed for a series of exposures upon a single plate. This enabled him to realize a long-standing ambition to visualize the wing shape of birds in flight (Marey, 1890). In a variant of this approach, Marey produced a series of images upon a single plate by reflecting the light path between exposures with a rotating mirror (Marey, 1888b). In this way, Marey generated images of a swimming eel from which he measured the speed of wave propagation, 45 years before Gray's report on the subject in JEB (Gray, 1933). Marey incorporated film into his camera designs as it became available near the turn of the century, which aided in a series of studies on human locomotion and flow visualization. By this time, George Eastman and others were manufacturing film for consumers, which provided the basis for a consumer market for photography and the technological foundation for the movie industry (Braun, 1992).
Recordings
Despite the precocious efforts by Marey, the use of film was not broadly adopted for biological investigation until manufacturers began developing cameras that could be mounted to microscopes. Progress in this area can be traced through the early cytology work of James Gray, initiated after his service in the First World War (Lissmann, 1978). Early examples of his studies on ciliary motion describe shape changes with hand-drawn illustrations without a time scale (Gray, 1922), whereas later work featured temporally resolved motion with supplemental photographs (Gray, 1930). Gray generated kinematics by modifying his microscope, which included an integrated movie camera that recorded at an inconsistent frame rate (Gray, 1930). Gray had a hole drilled into the camera's body, into which he directed a microscope objective that focused a spot of light onto the moving film during a movie recording. By interrupting the light path at a known interval, he was able to insert a strip of white lines that coded for a known duration to the side of the microscope images. Gray then applied this technique, no doubt with a change in lenses, in his study on the kinematics of fish swimming, published in JEB, which was his first performed at a macroscopic scale (Gray, 1933) (Fig. 2A).
In the early film era, a number of investigators worked around the limited frame rates of cameras by using strobe lights. Strobe lights deliver pulses of illumination at a controlled frequency and duration. Oscillatory motion at the same frequency as the strobe appears static because the subject is illuminated at the same phase in the cycle upon each burst of light. Therefore, one can determine the frequency of a rapid oscillation by adjusting the frequency of the strobe until the subject appears motionless. Strobes were initially devised for this purpose using an arrangement similar to Marey's rotary camera shutter, but with the perforations allowing light to escape a light box and thereby illuminate the subject. This technique was employed in JEB studies initially to characterize such phenomena as the 17 Hz motion of beating in copepods (Cannon, 1928). However, the invention of stroboscopic lamps (Edgerton, 1932) allowed for greater control of the flash frequency and synchronization with camera exposures, with the ‘Strobotac’ model (General Radio Company, Massachusetts, USA) the most popular to JEB investigators. The Strobotac was used at this time to resolve frequency while images of the blurred motion captured by still photos provided the amplitude of beating in insect wings, cilia and flagella (e.g. Sleigh, 1956; Vogel, 1966; Gray, 1955). Numerous JEB studies at this time used a stroboscopic lamp to illuminate swimming spermatozoa that were recorded through multiple exposures with a still camera (e.g. Brokaw, 1965, 1966), in a manner reminiscent of Marey's chronophotography.
Strobe lights were helpful to the development of early high-speed ciné cameras. In a classic JEB study on the kinematics of bird flight, R. H. J. Brown modified an old hand-cranked 35 mm movie camera to perform high-speed filming at up to 100 frames s−1 (Brown, 1948). He installed a motorized drive, complete with a clutch and brake to minimize the time to start and stop the system and thereby save film. To allow the drive to pull the film fast enough, he removed the mechanical shutter and intermittent mechanism. This device would have yielded blurry images under continuous illumination, because of the motion of the film. However, Brown used a high-speed strobe that would flash for a duration of ∼1 μs, which was brief compared with the film speed. To synchronize the strobe with the film, Brown installed three rotary switches along the drive shaft of the motor that coordinated the charging and triggering of the strobe in perfect synchrony with the motion of the film.
In the 20 years following Brown's study, high-speed ciné cameras became commercially available. This included the 16 mm Wollensak Fastax camera that recorded up to 8000 frames s−1 to study the explosive motion of a jumping flea (Bennet-Clark and Lucey, 1967). These high-speed cameras incorporated a time marker in the frame margins, similar to what Gray had installed in prior years. Therefore, commercial high-speed ciné cameras offered excellent temporal and spatial resolution, even by contemporary standards.
Data acquisition
Kinematic measurements from film recordings were labor intensive. Description and depiction of morphology quickly gave way to marking and tracking the movement of morphological landmarks through time. Methods remained ad hoc, but typically involved projecting a ciné film image onto a grid, followed by different methods for recording coordinates from that grid. For a series of still images (e.g. Fig. 2A), one could mark coordinates upon a layer of transparency paper placed upon a still photograph and then resolve the positional changes at the scale of the subject, provided there was an object of known distance within the frame or in a separate photo under the same magnification. By the 1980s, investigators employed computer digitizing tablets, sometimes with custom software, to digitally select coordinates from projected film images (e.g. Delcomyn, 1987; Wassersug and Hoff, 1985; Videler and Hess, 1984). All these methods demanded a substantial amount of human effort, a requirement that may have limited the size of kinematics datasets. This limitation of the manual selection of landmarks was not easily circumvented by subsequent technologies either.
The analog video era (from the 1980s to the 2000s)
Technological origins
Analog video originated in the world of broadcast television and began to have practical applications to scientists by the late 1980s. A television image was generated by a cathode-ray tube (CRT), which controlled the projection of an electron beam upon a phosphorescent surface that was positioned behind a glass viewing screen. The surface was activated to glow more brightly when the electron beam was powered at higher intensity and its position was controlled with electromagnetic coils. These coils moved the beam rapidly in horizontal lines from left to right, with lines starting at the top of the screen and then working their way down, like words on a page. The beam varied its light intensity all along the way to generate an image. This scanning was performed in two passes, each called a ‘field’ that consisted of 240 visible lines, and the projection of the two fields effectively doubled the spatial resolution. The fields were captured at 60 Hz in the USA (50 Hz in Europe), but displayed at 30 Hz (or 25 Hz in Europe). Interlacing provided an illusion of capturing faster motion by visually averaging events captured at 60 Hz and played back at half that rate (Lipton, 2021).
Although CRT televisions became common in households in the US and UK in the 1950s, the video cameras of that era were generally impractical for scientific applications. Broadcast cameras were based on analog camera-tube technology and hence were large, heavy and expensive. In addition, movie recording initially entailed filming the broadcast by directing a film camera at a CRT, and thus offered little advantage for an investigator seeking to avoid the time and cost of film developing. It was not until the advent of the consumer video-cassette recorders (VCRs) and solid-state image sensors that the advantages of video for science emerged. Unlike cumbersome camera tubes, solid-state image sensors generate images from a thin two-dimensional array of photo-sensitive pixels. These sensors, beginning with the charge-coupled device (CCD), generated the opportunity for the design and marketing of consumer-grade video cameras that permitted one to record their own movies that could be displayed on a CRT. Such cameras were widely available on the consumer market by 1990 (Lipton, 2021).
Recordings
Video presented advantages for investigators, despite its spatial resolution being inferior to film. The cassette tapes were relatively affordable and reusable and allowed for an ease of recording over a long duration. The ability to instantly review video offered experimentalists a valuable means to modify the lighting, and perhaps the experimental design, in the midst of conducting the experiments. However, analog video required new methods for acquiring coordinates from recorded images. To make use of existing techniques, some investigators combined video recording with simultaneous filming. This approach allowed the use of video to identify the durations of film for acquisition (e.g. Tang and Wardle, 1992). A variant of this approach had investigators film a CRT screen of only the portions to be analyzed with a 16 mm camera (e.g. Delcomyn, 1987). Before the widespread use of high-speed analog video, short-duration high-speed ciné recordings might be coupled with the long-duration recording provided by regular-speed video (e.g. Graham et al., 1987). Therefore, investigators were beginning to find creative ways of incorporating low-resolution analog video into their workflows, while still retaining high-resolution film as the primary means for kinematic measurements.
By the 1990s, high-speed analog video cameras came to supplant ciné for measuring rapid motion. This occurred a decade after NAC Image Technology had developed its first high-speed video camera for industrial research. Using VHS or S-VHS cassette, these systems initially recorded at 200 fields s−1, but models offering rates of 400, 500 and 1000 fields s−1 were available and routinely marketed to investigators by the early 1990s (nacinc.com). To compensate for their low light sensitivity, the NAC and Kodak (manufactured by Photron) systems were sold with a bright synchronized strobe light. These innovations allowed the analog video era to continue the growing emphasis on studying high-speed phenomena that had begun with high-speed ciné (Fig. 1).
Data acquisition
The workflow for acquiring kinematics from analog video entailed a combination of analog and digital technologies. Video cameras first collected a digital representation via a CCD, then converted the resulting image to analog format for recording on tape. Portions of these analog videos were then selected by the investigator for conversion back to digital format during computer-based analysis. Although cumbersome to describe, in practice this workflow combined compact cameras with a widely supported tape storage technology that was capable of recording continuously through long-duration experiments.
Investigators and small companies addressed the problem of acquiring coordinates with a computer. Computers by this time could be adapted for digitizing video with the addition of a frame-grabbing card, which converted an analog video input to a digital image. Coordinates were selected with software developed by the investigator (e.g. Jayne and Lauder, 1995; Crenshaw, 1991) or NIH ImageJ (e.g. Quillin, 1999; Schneider et al., 2012). This workflow generally included deinterlacing the images to separate the two 240-line fields in each frame (Fig. 2B,C), which served to double the temporal resolution of standard VHS video to 60 Hz (50 Hz in Europe). A popular turn-key system was developed by Peak Performance Technologies, which included some rudimentary autotracking of high-contrast markers as well (e.g. Ritter and Nishikawa, 1995; Robertson and Johnson, 1993).
Analyzing video recordings tested the limits of computer technology available at the time. Beyond the required installation of a frame-grabber analog-to-digital converter board, video files needed to be saved to disk and individual frames copied to RAM for display and analysis. A frame of standard definition video (S.D.640×480 pixels) required about 300 kB in 8-bit color. For reference, the original Macintosh computer debuted in 1984 with a monochromatic display, 128 kB of RAM, no hard drive, and a disk drive for floppy disks that could store 400 kB of data. It is therefore unsurprising that NAC high-speed cameras did not make a significant appearance in JEB until around 1990, when RAM shipped in computers at levels exceeding 8 MB (Fig. 3A). Other factors, such as the speed of central and graphics processors were additional crucial factors to enable video data processing by computer. By the early 1990s, these hardware components where of sufficient capacity that the desktop software could load and manipulate video. For example, the video-editing software Adobe Premiere launched in 1991 (Lipton, 2021).
The digital video era (from the 2000s to the present)
Technological origins
The technological transition from ciné film to analog video represented a fundamental shift from chemical to electronic detection and recording of light. The transition from analog to digital video contains no such shift and, from a technological point of view, proceeded in piecemeal fashion. As described above, by the middle of the analog video era, image data were first digitally detected using a CCD, then stored in analog form on tape, displayed in analog on a CRT screen, and finally converted back to digital form for analysis. Later in this era, digital video tape formats appeared that moved the storage encoding to a digital format while continuing to use tape as the medium. Thus, the analog era was actually digital in many of its internals by the early 1990s, and by the end of the decade, commercially available high-speed analog video cameras were capable of recording at more than 1000 frames s−1, yet still used standard VHS tape for storage. In the face of this piecemeal transition, we mark the shift to the digital video era as the point where data were collected, stored and analyzed using digital technology without any intermediate analog steps. In general terms, this came about once personal computer volatile and non-volatile storage (i.e. RAM and hard disk) were large enough for video display and especially storage (Fig. 3A).
As with many other technological advances described here, a few pioneering researchers with specific needs and appropriate skill-sets developed custom fully digital workflows well before the appearance of ‘off the shelf’ equipment with the same capabilities. For example, some groundbreaking early experiments directly coupled a computer frame-grabber board to a CCD camera and used custom computer programs to analyze the live video image to extract relevant kinematic parameters (Godden and Graham, 1983; Dean, 1991; Winberg et al., 1993; Schurmann and Steffensen, 1994). Because the images were processed as they were acquired, it was not necessary to store the videos, which worked around the absence at the time of low-cost and high-capacity digital storage media. As these studies demonstrate, a purely digital workflow was possible with computer technology (i.e. an Apple II) as early as 1983 and the data could be generated instantaneously. However, for most biomechanics research questions, such real-time video analysis was infeasible and analog tape fed by CCD sensors remained the dominant technology. Interestingly, real-time video workflows still have a place in current research (e.g. Fry et al., 2009), where they enable closed-loop experiments on animal sensing and control of movement.
Recordings
The rapid growth in digital storage technology through the 1990s (Fig. 3A) eventually made fully digital data acquisition and workflows widely accessible. The introduction of commercial high-speed digital video systems, such as the RedLake PCI-500 and the Photron 1024, provided further impetus to the shift and continued the emphasis on recording rapid behaviors (Fig. 1). These fully digital workflows further reduced the size of the camera apparatus and freed video technology from the scanline resolution limits required for compatibility with VHS and DV tape formats. Unlike contemporary analog tape systems, these initial digital video cameras could not record for long durations, but the combination of post-event triggering and rapid review of the video sufficiently compensated for this lack in most cases and the use of high-speed analog tape systems declined rapidly (Fig. 3B).
The transition to the digital era served as the basis for further specialized development of kinematics recording methods and technology. For example, removing the need for a mechanical shutter to expose film or rolling tape past a record head for storage allowed for increases in maximum frame rates. The Photron PCI-1024 from the early years of the digital video era could achieve limited resolution frame rates in excess of 50,000 Hz. These new capabilities enabled measurement of the kinematics of ever faster movements such as ballistic spore and seed launches by fungi and plants (e.g. Yafetto et al., 2008), and escape and prey acquisition movements powered by elastically stored energy (e.g. Patek et al., 2006). Flexibility of use and recording speeds were hallmarks of the digital transition, and coupling digital high-speed cameras with X-ray imaging produced the XROMM workflow (Brainerd et al., 2010), with powerful results for directly measuring internal movements and structures not seen by visible spectrum cameras (e.g. Kambic et al., 2014; Menegaz et al., 2015). These capabilities opened new areas for investigation, especially during activities such as feeding and breathing that involve many interacting skeletal elements not easily tracked by external markers (Fig. 3C).
Data acquisition
Image analysis in the digital video era initially operated in the same framework as the analog video era, with specialized workstations and software from vendors such as Peak Performance. However, digital video analysis no longer required the specialized frame-grabber hardware used for analog video, and standard digital video formats facilitated the creation of many small software packages for analysis. At the same time, the open-source software movement was gathering momentum while increasing computer performance combined with the growing size of kinematics datasets pushed researchers toward using ‘scripting’ or non-compiled programming languages such as Python and MATLAB for data analysis. The DLTdv package for MATLAB (Hedrick, 2008) combined many of these trends, along with the move from 2D to 3D kinematic analysis, and became a popular option for computer-based digital video analysis (Fig. 4).
Portable consumer high-speed cameras
The high-speed digital video cameras available in the year 2000 typically combined electrical power demands on the order of 100 W, with limited record time and large storage requirements relative to contemporary hard drives, and needed a desktop computer to control them. The evolution of these designs in the following decade produced continuous improvement in image resolution, light sensitivity, maximum recording duration, and other parameters, but left the core concept of costly, special-purpose cameras controlled by a powerful host computer unchanged. However, by the year 2012, continuous technological advancement in digital camera and video technology led to the introduction of nascent high-speed recording capabilities in consumer-oriented products, such as the GoPro Hero series. These rugged, highly portable, battery-powered cameras record directly to on-camera storage using video-compression algorithms implemented in specialized hardware. Subsequent advancement in capabilities driven by consumer demand has since produced improvements in image resolution and quality, although frame rates beyond the approximate 500 frames s−1 required to ‘freeze’ human motion may not be forthcoming.
These portable cameras have brought high-speed digital video kinematics to field conditions where the costly industrial or scientific high-speed cameras could not reach, and lab situations where those solutions were too cumbersome or costly. Indeed, even most contemporary smartphones include high-speed recording capability, facilitating ad hoc high-speed recording of many lab and field phenomena. The small size of these cameras, coupled with simultaneous technological improvements to unmanned aircraft, have even made drone-based video kinematics a reality (Basu et al., 2019). However, the consumer origin of these cameras means that synchronization of the recordings with other cameras or scientific apparatus remains a challenge, and thus far these and other issues have prevented consumer-oriented cameras from supplanting industrial and scientific high-speed cameras, even in situations where the necessary frame rates are available (Fig. 3).
The continuous recording capabilities of consumer-grade cameras also challenge digital video analysis workflows based on semi-automated or manual tracking of landmarks because of the large quantity of video that can be collected. Advances in neural network-based automatic image analysis (e.g. Krizhevsky et al., 2017; Mathis et al., 2018) suggest a path ahead for dealing with ever greater quantities of image analysis, but the practical impact of these methods on research published at the present time has yet to fully emerge and most papers mentioning these terms are methodological rather than research publications. In many of the technological changes described here, we identified early pioneers who, because of specific needs or unusual skill-sets, adopted a technology many years before it became sufficiently easy to use and well understood for widespread application. In our opinion, machine learning or artificial intelligence (AI)-based image analysis workflows still fall in this stage and require further improvements in usability along with a widely applicable set of best practices to achieve the potential revealed by early adopters.
Kinematics without image capture
The ubiquity of camera-based kinematics measurements speaks to their broad utility, but its limitations have pushed many researchers to seek alternatives. As a first limitation, camera-based kinematics requires substantial post-processing to extract numerical measurement from recordings. The time and resources required for these secondary steps might be unavailable, depending on the scale required, or simply better put to use on other aspects of the research project. This has led to a diversity of methods over the years that measure motion or movement directly, often as a series of time-varying voltages rather than an image series.
Laboratory methods
Early examples of kinematics without cameras included kymographs that were mechanically coupled to the animal. These recordings would be made with a smoke drum, which is a sheet of paper covered with a thin layer of soot, wrapped around a cylinder. A motor would drive the rotation of the cylinder at a fixed rate, which allowed for recording the up-and-down motion of a stylus along the height of the cylinder. The stylus would generally be coupled to a lever that was actuated by the pulling of a thread, which might be tied to a portion of the body wall of a sea anemone or worm (Batham and Pantin, 1950; Lawry, 1966). Therefore, the tension generated by the body wall would pull on the thread and thereby move the stylus to leave a trace in the soot of the rotating smoke drum. This approach succeeded in generating a real-time graph of animal motion that might be synchronized with other motion or physiological recordings.
Subsequent approaches employed electronics for real-time recordings. For example, Schilstra and Hateren (1999) used a set of three miniature magnetic sensor coils to track the 3D position and orientation of the head and thorax of blowflies during free flight. This produced measurements of relative head movement that led to new insights into how flies stabilize their vision during rapid flight maneuvers. Similar camera-based measurements appeared more than a decade later (e.g. Boeddeker and Hemmi, 2010). More prosaically, the simple combination of a photodetector and light source provided direct measurement of the flapping frequency and amplitude of tethered insects in a vast number of insect flight biomechanics and neurobiology studies (e.g. Götz, 1987; Dickinson et al., 1993). Even more simply, the flight tone produced by the flapping wings of insects directly reported one crucial aspect of kinematics – flapping frequency – without requiring any further analysis steps (e.g. Dorsett, 1962). All of these approaches provide direct measurement of animal movements, but were limited in their generality compared with image-capture kinematics and usually only applied to a few specialized research areas.
Cameras that automatically identify and track specialized markers attached to the study subject have long been popular in human biomechanics and gait studies. Such systems, such as those manufactured by Vicon Systems, offer an alternative solution to the data-acquisition problem posed by image-based kinematics, and have been included in JEB studies (e.g. Hallemans et al., 2004). Use in comparative studies is more rare, but automatic motion capture systems have been applied to animals as diverse as infant baboons (Raichlen, 2006) and agamid lizards (Metzger, 2009). Despite their capacity for rapid data acquisition, these systems have not displaced image-capture measurements in the JEB literature. This may be due to factors including cost, ease of use, and the absence of the additional contextual information captured by images.
Field methods
A limitation of camera-based kinematics is the requirement that the animal remain in view of the cameras during the experiment. This can pose substantial challenges for animals moving freely in natural environments. Cameras can be used in such circumstances, and even calibrated for 3D recording in outdoor volumes much greater in size than indoor laboratories (e.g. Theriault et al., 2014; de Margerie et al., 2015). However, these calibrated volumes remain a fraction of many natural ranges. On-animal devices that measure kinematics can address this challenge, at least when coupled with modern digital electronics for data-logging. At the present time, the most common technologies are satellite navigation receivers, which directly measure position on the planet along with accelerometers and inertial measurement units (IMUs), which directly measure the second derivative of motion along with gravitational acceleration. Image-capture kinematics commonly uses numerical methods to measure acceleration, which is often of greater research interest than the underlying position time series. Therefore, IMUs offer the tantalizing possibility of directly measuring the quantities of greatest interest in laboratory and field biomechanics studies.
Accelerometers first appeared in the pages of JEB as an option for measuring animal biomechanics in 1978 when a 2-axis unit was attached to the tail of swimming bluefish in a laboratory experiment (Dubois and Ogilvy, 1978). However, these early devices were limited by their several-millimeter size and dependence on wired connections to external devices for power and recording. Nevertheless, accelerometers occasionally appeared in experiments where their properties – direct measurement of acceleration, high sampling frequency and independence from light – were particularly valuable (e.g. Schilling and Carrier, 2010). Many of these studies also collected video to help interpret the accelerometer output and, in extreme cases, a complete high-speed video analysis was used to provide the orientation information needed to analyze the accelerometer records (e.g. Hedrick et al., 2004). Subsequent innovations in electronics have greatly enhanced the utility of IMUs for studies of comparative biomechanics. The commercial availability of MEMS (micro-electrical mechanical systems) accelerometers based on semiconductor chip fabrication technologies began around the year 2000. These devices greatly reduced accelerometer size, power requirements and cost, which thereby facilitated their use in animal experiments. Accelerometers quickly became a sensor of choice for animal-borne data loggers that permit measurement of biomechanically relevant information from free-ranging animals. While initially applied to larger-bodied swimmers (e.g. Yoda et al., 1999), continued technical improvements have reduced the package size and weight sufficiently for use with flying birds (e.g. Kogure et al., 2016). At the present time, accelerometers are a tool of choice for on-animal movement tracking and their development has opened up many new areas of research in animal locomotion and movement ecology. These devices allow collection of long-duration field records that can show how classical biomechanical phenomena, previously studied in the laboratory, fit into the daily lives of free-ranging animals.
Effects of technology on kinematic measurements
We have detailed a number of studies that represent trail-blazing methods for their time. These examples include technical innovations that were later adopted more broadly. The idea behind annotating a ciné recording with a calibrated time code may be traced back to Marey (Braun, 1992), and implementing this technique did require a hack to commercially manufactured equipment on the part of James Gray (Gray, 1930, 1933). Subsequent cameras would be manufactured with time-coding devices to ensure temporal accuracy, which was especially essential for high-speed recordings that required durations to accelerate and decelerate the film (e.g. Bennet-Clark and Lucey, 1967). It was often not until such innovations were incorporated into manufactured products that they appeared on the pages of JEB in appreciable numbers. Such was the case for Brown's precocious ability to attain 100 frames s−1 by electro-mechanically synchronizing a stroke light with a ciné camera (Brown, 1948). It was only years later that high-speed recordings would be broadly adopted, once they could be purchased for the industrial market (Fig. 1). Innovations that were not commercialized may have never received broad adoption in JEB. For example, we described a real-time image-processing technique from an article published almost 40 years ago (Godden and Graham, 1983) that was not directly commercialized, and when similar capabilities did appear in commercial motion-capture cameras, the requirements for reliable operation were largely specific to human experiments. Thus, trail-blazing techniques failed to be broadly adopted because they did not offer substantially improved benefits and/or they were too technically challenging for most investigators to implement. By our reading, the best prospect for the adoption of a novel innovation occurred when they were included in the design of manufactured products and this required that the innovation offered a benefit to a commercial or industrial market.
Our survey of JEB articles has revealed a trend in the focus on high-speed kinematics. Once high-speed ciné cameras became broadly available in the 1960s, the growth of articles mentioning the words ‘high speed’ outpaced the growth in the number of JEB articles, until perhaps the 2000s (Fig. 1). This trend mirrors anecdotal evidence gleaned from the early volumes. Common subjects in the first 3 decades of JEB included studies on snails (Lissmann, 1945), worms (Chapman, 1950) and sea stars (Kerkut, 1953). The next 30 years included kinematics on the swimmerets of lobsters (Davis, 1968), the detailed motion of an insect wing beat (Wood, 1970) and the jumping of locusts (Bennet-Clark, 1975). We therefore find it likely that the behaviors and taxonomic groups selected for study were influenced by the commercial availability of the faster cameras over the course of over the film era. The persistence of studies on high-speed phenomena through the decline in use of ciné cameras (Fig. 1C) was enabled by innovations in analog and digital cameras (Figs 2 and 3). Throughout the eras, enhancement in temporal resolution has facilitated time-resolved motion and enabled higher accuracy in measures of maximum acceleration and velocity (Walker, 1998).
Developments in computer technology from the 1980s to the 2000s offer perhaps the best illustration of the ways in which technology can constrain or enable scientific progress in the area of kinematics. NAC introduced their analog high-speed camera systems in 1980, which was years before computers were capable of loading video frames into memory for analysis (Fig. 3A). Although the recordings from NAC cameras may have been useful to easily visualize manufacturing machines, crash-test dummies and explosions, the inability to measure kinematics from their videos offered an inferior technology to high-speed ciné for JEB investigators (Fig. 3B). By the 1980s, digitizing tablets allowed the capture of coordinates from projected film images with modest requirements for computer memory and storage. However, the benefits of analog video to easily review recordings of long duration began to find appreciation once consumer desktop computers were capable of digitizing, storing and rendering standard-definition video in the 1990s. Further developments in computing allowed for an easy embrace of a purely digital workflow, once vendors were capable of manufacturing them. At the present time, it appears that AI-based automation of image analysis is poised to greatly speed up kinematics data acquisition workflows, but the technical challenges required to achieve high-quality results with these tools appear to have precluded their widespread use, thus far. We expect further productization and/or commercial support for these capabilities to change that situation in the next few years.
Our analysis of the JEB literature included an assessment of the size of datasets from a random selection of studies that included kinematics (see Supplementary Materials and Methods for details). Only about one-tenth of the variation in the number of individuals may be explained by the year of the study (Fig. 5A). The number of landmarks and the number of experiments per individual did not individually offer significant relationships with time (Fig. 5A–C). However, the product of all of these quantities provides an overall metric of the size of datasets, which we found to have a significant positive trend over time by linear regression (Fig. 5D). This trend accounts for only one-quarter of the variation in dataset size, but the average size has increased by a factor of 36.8 over the past century. This trend does not appear to change between the film, analog and digital eras and it is therefore likely that the increase in datasets may be as much a response to changes in statistical standards as technology. One might expect that this enhancement in dataset size would prompt investigators to adopt a more comparative approach. However, we did not find this to be the case, as the number of species within a study did not significantly increase with time (Fig. 5E). The vast majority of studies have focused on an individual species, across eras. It is therefore likely that the choice about the number of species is not constrained by technology, but is rather a choice by JEB investigators based on interest, or bias in approach.
Conclusion
A century of research in comparative biomechanics has evolved in a manner aided by technological developments. We have not uncovered evidence of punctuated progress, where a technology has resulted in a step-change in the scale of research either within articles (Fig. 5D) or through the number of articles (Fig. 1). Instead, incremental improvements in resolution, frame rates, recording duration and computing power have permitted gradual changes in research that have spanned technological eras to meet the rising standards of rigorous science. Although a new investigator entering this field is far more likely to learn a programming language than to enter a dark room to develop a reel of film, kinematic measurements are likely as fundamental to their science as they were to the young James Gray.
Acknowledgements
Our review of the literature was assisted by Fion Li (UC Irvine).
Footnotes
Funding
M.J.M. is supported by grants from the National Science Foundation (IOS-2034043) and Office of Naval Research (N00014-19-1-2035, N00014-22-1-2655); T.L.H. is supported by National Science Foundation (IOS-1930886).
References
Competing interests
The authors declare no competing or financial interests.