Three-Dimensional Reconstruction of Animal Flight Paths and the Turning Flight of Microchiropteran Bats

Still and ciné-photography have been widely used to investigate the rapid wing and body movements of flying animals. If only a single camera is used, however, the film image is a plane projection of a complex three-dimensional object, and it is rarely possible to obtain more than the most rudimentary three-dimensional information about the structure of the object or its movements in time. To describe body deformation more exactly two approaches have been used: either a geometrical relationship is imposed between parts of the object so that structural information can be gleaned from a single image, or multiple images from noncoincident optical axes are used. In the first approach, a distance (for instance that between the wingtip and the wingroot) is assumed not to vary, and the method is restricted to animals where this approximation is acceptable: it is unlikely to be appropriate, for instance, in studying slow flight in birds and bats where wing deformation is both substantial and important. The second approach has the disadvantage that experiments with multiple ciné cameras are complex and costly, while with a single camera and a beam-splitting system of mirrors there is inevitable degradation of image quality. The method presented in this paper attempts to avoid some of these problems. Although the method has some similarities with these techniques, it is simplified considerably: the use of nonmetric 35 mm cameras and of a geometrically simple test object permits significant economies in cost and in analysis time. In this application to flight manoeuvres in bats, two motorized 35 mm cameras are arranged on approximately perpendicular axes; the shutters of the camera are held open for a suitable length of time, and a sequence of images of the moving animal is obtained on a single frame with a stroboscope or a series of electronic flash units (Fig. 1). Camera configuration and image translation, rotation and magnification are controlled externally by incorporating a test cube in the object: the complete structure of the object can be estimated from the digitized image by a microcomputer algorithm without the need to control or survey the positions and orientation of the cameras or the focal lengths of the lenses, or to employ metric cameras with fiducial marks permitting internal image calibration.

In outline, this approach to flight path analysis owes much to the similar technique with three cameras used by Etienne Marey (1890) in his studies of bird flight; Marey however did not attempt to reconstruct his images numerically. More recently, three-dimensional analyses of flying animals have employed stereo-photogrammetry with still cameras (Pennycuick, 1973; Spedding, 1982; Major & Dill, 1978; Spedding, Rayner & Pennycuick, 1984; Biesel, Butz & Nachtigall in Nachtigall, 1984) or have used either two ciné cameras or one ciné camera with an arrangement of mirrors to show multiple views (Magnan, Perriliat-Botonet & Girerd, 1938;Bilo, 1971, 1972; Norberg, 1976; Zarnack, 1977; Buelthoff, Poggio & Wehrhahn, 1980; Trappe, 1982; Wehrhahn, Poggio & Bülthoff, 1982; Zeil, 1983; Dahman & Zeil, 1984). Greenewalt (1955) analysed movements using a single still camera with a stroboscopic image, but was therefore restricted to analysing straight flights perpendicular to the camera axis.

Our development of this method was inspired by the need for an economical microcomputer-based method of monitoring gross body movements in slow-flying birds and bats; the method has also proved invaluable in quantifying wingbeat geometry and gait patterns in bats (Aldridge, 1985). In this application, the numerical algorithm is used to reconstruct the three-dimensional scene from the images of a moving animal seen by two still cameras, but the method is not restricted either to studies of movement or to the analysis of still photographs; it may also be used to reconstruct complex static images or to analyse movement sequences recorded on ciné film. (It must be pointed out, however, that if a beam splitter is used on a single camera it will probably be necessary to adopt more sophisticated photogrammetric techniques for image resection to allow for distortion introduced by the optical configuration.) The method is illustrated by a demonstration of the way in which it has been used to determine accurately the velocity and acceleration patterns and the forces acting during a slow powered turn by a small bat; these results have enabled us to clarify the mechanical constraints on turning performance and manoeuvrability in flying animals.

Various numerical algorithms are available for the three-dimensional reconstruction (resection) of an object viewed as a pair of images, each containing a number of points (a test pattern) whose real locations may or may not be known. These methods generally use least-squares error minimization, frequently combined with iterative convergence (Wolf, 1983). The resection problem has two phases, the first being to use the test pattern to relate the position of the camera(s) to a known reference frame, and the second to obtain the spatial coordinates of points visible in both images. The resection of a single view with a known (surveyed) test pattern comprising at least three points has been outlined by Wolf (1983) and Dahmen & Zeil (1984). With two cameras, the camera positions may be reconstructed by iteration if five points are visible in each image, while if eight points are visible, a solution by linear simultaneous equations is possible (Longuet-Higgins, 1981). However, in our experience these methods suffer from major limitations when implemented on a microcomputer; in particular they make excessive demands on memory size, and can require the manipulation of a large, sparse matrix, a process which — despite the development of various sophisticated numerical techniques –can be highly sensitive to the numerical errors inherent in the use of microcomputers, and which can make the entire calculation unreliable.

We have therefore derived a method which is fully practicable on a microcomputer, and which avoids these numerical problems by using the vertices of a cube of known size as test pattern. From the cube image the camera position may be estimated by classical geometric methods [the resection cannot be solved by Longuet-Higgin’s (1981) matrix method]. This approach has the additional important advantages that the camera and lenses need not be calibrated since the cube allows all image reconstruction to be performed externally to the cameras, and that since the resection is computed independently for the two images the cameras and lenses used need not match. Ideally the cube should be large enough so that its image fills the film frame in each camera. The most severe limitation on the geometry is imposed by the simplifications adopted here for the cube resection: the camera axes should be close to perpendicular to the cube faces, and the principal axis (line of sight) of each camera should pass close to the cube centre; in each case the discrepancy should not exceed 10°. If this condition is not met the resection must determine the angular orientation in addition to the location of the cameras and their line of sight, and more complicated iterative procedures should be used.

The cameras used for these experiments were Nikon F2, FM and FM2 with motor drives and 55 mm f2·8 and f3·5 Micro-Nikkor lenses; these lenses are particularly free from distortion (Spedding, 1982). Illumination was provided by one or two Dawe model 10203C ‘Strobosun’ stroboscopes, each producing 40 W output at flash rates between 10 and 200 Hz. The cameras were usually configured so that as the bat flew level along the ŷ axis, the cameras viewed normal to the flight path along the and axes: the geometry and other features of the turning flight experiment are shown in Fig. 1.

The images were analysed on a Research Machines 380Z microcomputer with 56 kbytes memory, running a standard version of BASIC (RML version 5·0L). Data were input to the computer through a Summagraphics Bit-Pad One digitizing tablet. The images used varied between projections of slides or negatives, tracings from an enlarger or a camera lucida, or photographic prints. Ideally the full area (approx. 0·25 m square) of the tablet should be used to minimize errors, the typical digitizing uncertainty being of the order of 0 25 mm. The computer algorithm corrects each image for scale and orientation, and it is necessary to control for rotation or magnification either on the tablet or in preparation of the image. Results were output either on the computer’s high-resolution graphics display or on a graph-plotter (Hewlett-Packard 7225A).

Any comparable microcomputer may be used in place of the 380Z; the major requirement is sufficient memory capacity (a minimum of 45 kbytes in addition to the BASIC language) to accommodate the programmes and data; this memory size is sufficient for the analysis of up to about 80 object points. A simplified version of the analysis programme, calculating and displaying velocity and acceleration in two dimensions, has also been implemented successfully on Commodore PET 4032 and BBC model B microcomputers.

The resection is caclulated independently for each image, and gives the locations of the cameras together with scale and rotation factors which encompass all image translation between the film plane and the computer. The geometry and notation are summarized in Fig. 2. Object coordinates are expressed relative to an origin at the cube centre. In the following description of the resection for a single camera the reference axes are defined so that the camera views parallel to the axis in the − x direction; the principal axis or line of sight of the camera is assumed to be approximately normal to the yz faces of the cube, and the camera focus, an object point and its image point in the apparent film plane are assumed to be collinear, so that lens and image distortion are neglected (Wolf, 1983). The y and z axes are defined to be perpendicular to the other faces of the cube; the second camera views similarly along one of the other axes, and the same algorithm is used to resect that image with a suitable change of notation. Primed quantities are expressed in digitized or computer coordinates; as the effective foçal length is unknown the location of the apparent film plane is arbitrary, and for simplicity is defined so that these primed quantities have the same dimensions as object points. The superscript ^* denotes projections of vectors perpendicular to the axis ⁠, the x component being, where necessary, specified independently. Film plane coordinates comprise two components only but the ^* is omitted; they are initially expressed relative to an arbitrary origin. The method is intended to take account of distortion of the image due to displacement of the camera location from the axis; additional sources of error arise if the principal axis is tilted. If the camera-object configuration satisfies simple geometric conditions these errors will be shown to be unimportant; methods by which the effect of tilt can be solved exactly have been given by Wolf (1983), and are not discussed in this paper.

When digitized coordinates of object points have been input to the computer independently for the two images, the solution is obtained by the following manipulations.

• Translate all image coordinates by addition of the quantity ⁠, where are the images of the cube vertices, so that image coordinates are expressed relative to the estimated image O′ of the cube centre O.

• Rotate the image about the axis through the new origin so that the mean of the angles between the eight cube edges lying perpendicular to the principal axis and the horizontal or vertical (as appropriate) is zero. This is the simplest means of orienting the digitized image, and -despite distortion introduced at this stage if the principal axis is tilted — has proved sufficiently robust. In all following calculations the image points are assumed to have been translated and rotated in this way.

• Obtain the distance, d, between the camera and the cube centre from the magnification of characteristic lengths In and If on the images of the near and far faces of the cube. To minimize errors the length must be the same on both faces, and should intersect with the principal axis; since initially this point is unknown the length is chosen first to be the mean of the two diagonals. By similar triangles, the x coordinate of the camera position is then given by:

(vi)The resection process (steps i–v) is then repeated for the second camera (in this application lying along the axis).

Resection has now determined the camera focus positions C₁ and C₂, and a set of datum points X_d1 and X_d2 (respectively on the x = 0 and z = 0 planes) for each object point x as viewed by each camera. It is useful to determine at the time of the experiment the camera positions and scaling independently for comparison with values estimated by the resection. Once the camera locations and the datum points are determined no further reference need be made to the experimental configuration.

The performance of this resection and reconstruction method can be influenced by errors of two distinct types, the first due to performance of the resection algorithm, and the second to digitizing errors, to inaccuracies in construction of the test cube, to optical distortion (which we have assumed to be small) and to imprecision in determining the image locations to be digitized.

Resection errors have been assessed by analytical expansion of the formulae derived above and by computer simulation; as might be expected, errors are due largely to tilt of the principal axis. If the camera axis is normal to, but offset from, the cube centre with |C^*|<h, object points within or on the cube are reconstructed without iteration to an empirical accuracy of better than 0·25|C^*|²h / d², and these errors can be eliminated entirely if iteration is used. With the principal axis of one camera tilted through a small angle θ, the leading term of the error in estimation of datum points Xd is θ|x|²/dh, but the overall accuracy of object point reconstruction is rather better than this, the maximum error being found empirically to behave as 3·5θh^2·5/ d^1·5 at the edges of the cube; the error decreases considerably towards the principal axis. The majority of the distortion due to tilt has the effect of contracting or dilating the image in a plane normal to the tilt: for instance, if the principal axis of one camera is tilted from the axis in the xy plane through angle θ towards the Ŷaxis, then the z coordinate of a point x = (x,y,z) within the cube is overestimated by − θyz/d, but the x and y coordinates are accurate to within order of magnitude 0²h³/d². Accordingly, in a typical experimental configuration with d = 5h and with the relatively large tilt angle θ = 5°, the maximum error expected due to resection limitations is approximately 0·017h, or 0·0035 m with a cube of side 0·4m.

The overall performance of the experiments has been assessed by analysing a series of photographs of a test object comprising the resection cube together with a number of known test locations within, on and outside the cube. Dahmen & Zeil (1984) have suggested that the positional error in x can be estimated by the length e = (1 /2)|x₁ –X₂| (cf. equations 6a, 6b). Although this quantity forms a useful ad hoc estimate of object point errors, it does not measure the whole error and should be used with care. For instance, if either or both cameras are tilted within the plane of the two ideal camera axes, then for all points on that plane ŷ is zero, even though the resection may be wildly inaccurate if θh/d is at all large. Accordingly, we have preferred to assess the method by the overall reconstruction performance of the test object, which was photographed with different lenses from a range of positions and tilt angles. With moderate care in setting up the configuration, and with θh/d within 0·02 and |C^*|<0·5h, the maximum overall error -including the influence of cube construction, resection and digitizing inaccuracy — obtained for the cube vertices in successive experimental trials was 0·03h. The cube centre and the centres of the xz faces were reconstructed consistently to within 0·015h, with no apparent systematic variation within the cube. There was no apparent bias according to camera location, and with a cube edge of 0·4 m the maximum absolute error was 0·007 m for an object point lying within the cube and close to the ŷ axis. On the scale of a bat of at least 0·2 m wingspan, we consider this degree of accuracy fully acceptable, particularly since much of this error is due to digitizing inaccuracies and is therefore random, and may be treated by the methods given below. The procedure is least reliable along the edge of the cube closest to both cameras, since here solutions of the collinearity equations are least accurate, but within a radius of 2h from the cube centre reconstruction of the position of known object points was accurate to within 0·04h. Within the cube, resection errors of the order 0h/d are comparable to the other sources of error; we consider the method to be fully acceptable provided that θh /d is less than 0·02. In practice, there is little problem in constructing an accurate test cube, and with the cube and a spirit level or plumb line tilt angles of 1 or 2° are easy to achieve. More complex photogrammetric methods can increase significantly the accuracy of resection, but the technique proposed here has the great advantage of simplicity, and in practice its accuracy is limited as much by the resolution of the film negatives and of the digitizing tablet and its user as it is by the simplifying geometric assumptions which this resection method adopts.

The second phase of the analysis of flight paths comprises the estimation of velocity and acceleration and of other geometrical and mechanical quantities for the moving animal. These remarks are presented in the context of space resection with time information obtained by stroboscopic lighting, but they apply equally to the analysis of time series of position data obtained by any means. We consider it helpful to summarize here methods and formulae for the analysis of position data, some of which are otherwise not readily available. In all of these techniques, the vector components of position are treated independently. We consider a time series consisting of n position records, p_i, obtained empirically at equal time interval t.

Even though smoothing and the use of the least-squares parabola make sensible allowance for random errors in the position trace, it is still common for smallamplitude noise in position to be amplified and to distort the acceleration trace substantially and to render it useless. An alternative technique is provided by the Fourier series method of Lanczos (1957) and Hatze (1981), in which the position sequence is replaced by a Fourier expansion, and a cut-off frequency is determined to smooth the curve; the Fourier series is then differentiated either algebraically or empirically. However, this method demands that acceleration must be identically zero at the start and finish of the trace since this is an unavoidable feature of the position series. If the initial acceleration is arbitrary -as it must be in a brief segment of a flight path — we know of no more reliable method for the numerical differentiation of an empirical position series than the least-squares parabola.

It is often necessary to estimate values at times intermediate to the sample intervals in order to draw a smooth curve, to locate a local maximum or minimum, or to estimate a value at a specified time. The simplest methods are the four- and five-point Lagrange polynomials, which fit cubic or quartic polynomials to points surrounding the interval. The appropriate formulae are widely cited (see, for example, Lanczos, 1957).

Full descriptions of the application of this technique to bird and bat flight will be published elsewhere (see, for example, Aldridge 1985). As an illustration of resection and path analysis methods a brief description is given here of the configuration of the flight path during a single, slow, powered turn of a long-eared bat Plecotus auritus (Vespertilionidae; body mass m = 0·0095kg, wingspan 0·255 m), based on photographs taken in captivity in a flight cage (Fig. 1). The animal approached the test cube from the −ŷ direction; its exit was blocked by a back-lit Perspex sheet on the plane y = 0·4m, and the bat turned within the cube through almost 180° (while flapping its wings) before flying off in approximately the−ŷ direction (Fig. 3). The turn was slow and gradual, as usual in powered turns of small microbats, and is representative of many which we have filmed and analysed using this technique. The bat was accustomed to the cage and the obstacle, and habitually responded with a turn of this kind. The flash rate of 20·0 Hz was not sufficiently large for the wingbeat frequency to be distinguished, and only the mid-point of the line between the two humeral joints was digitized; this point could be readily identified in all images without the need for artificial markers on the bat. In straight flight in this individual the wingbeat frequency was approximately 14 Hz, but in the turn it rose to between 16 and 18 Hz owing to the fall in air speed (compare Aldridge, 1985).

Parallel projections of the flight path onto the three coordinate planes are shown in Fig. 4, and traces of position, velocity, acceleration and energy against time in Fig. 5. Curves are drawn by interpolating at four points within each interval with a five-point Lagrange polynomial; this curve is not a spline function and the gradient is discontinuous at the data points, but it is adequate to indicate the trend of variation. In this particular pair of photographs the camera axis tilts were both less than 2°, and the reconstruction error of points at the edges of the cube was estimated to be less than 0·002 m; systematic linear variation in this error across the cube due to principal axis tilt is estimated to affect velocity and acceleration estimates by at most 2 %; random error could be eliminated by smoothing, but in this case this proved unnecessary.

At the start of the turn the bat was flying straight and slowly (1·60ms⁻¹); curvature (Fig. 6) developed gradually, and the animal decelerated during the turn to a minimum speed of 0·55 ms⁻¹ when the curvature was sharpest. In order to minimize the loss of mechanical energy, the bat gained height as it slowed during the approach. Throughout the turn the y component of acceleration was large and constant (approximately − 4ms⁻²); speed was still increasing at the end of the sequence, although the animal was descending sufficiently fast that overall it was losing mechanical energy. The minimum radius of curvature between 0·08 and 0·10 m was considerably smaller than the bat’s wing semi-span and was maintained for a period of 0·2 s (almost four wingbeats); the rate of change of curvature as the animal banked into the turn was more gradual than during the succeeding recovery, owing to the assistance of gravity while the animal was descending. The vertical acceleration fell rapidly from 2·0ms⁻¹ to − 3·2ms⁻² during the start of the turn, and was negative while the radius of the turn was less than 0·4 m; after the turn it rose up to, and probably at a later stage above, zero as the flight path levelled out. Although the wings generated lift throughout the turn, the negative vertical acceleration reveals that they did not produce sufficient vertical force to support the weight, probably because the wings must be banked to produce the lateral centripetal acceleration required to alter the flight path: this lateral acceleration (calculated as KV²) reached almost g/2 at the middle of the turn. However, during the turn the mean normal lift coefficient (calculated from the vector sum of weight and lateral acceleration and the instantaneous flight speed) did not change greatly, but reached its highest value at the sharpest point of the turn. Our interpretation of this is that at this slow turning speed lateral acceleration is obtained only at the expense of weight support: the total force normal to the flight path never exceeds 1·10 mg, and is about 0·8 mg during the middle of the turn, and the estimated maximum bank angle of 32° agrees well with body and wing roll visible in Fig. 3A. This constraint on turning performance is probably imposed by circulation generation on the animal’s wings, and the bat appears to be flying at (or near) maximum lift coefficient. The approach air speed is slow compared to the normal speeds for straight flight in this cage for this individual (1·5–3 ms⁻¹), and if it were able to obtain a higher lift coefficient it is likely that the bat would either not decelerate so much or would climb less steeply, but would turn at similar turning radius with higher speed or would turn more tightly. It seems that the bat cannot improve turning performance by increasing lift coefficient much beyond the value in steady level flight; however the possibility cannot be ruled out that sensory performance and the animal’s own experience of the boundaries of the cage also constrain turning ability.

The resection method permits accurate estimation of mechanical forces and accelerations during the turn in a way which would be impossible with normal cinematography. As we have shown, the entire manoeuvre is tightly constrained by mechanical factors: the bat must fly slowly so that the lateral acceleration needed to turn within the cage is not unacceptably large; the cage width is 0·61 m, but the maximum turning diameter is in this case about 0·25 m since the approach flight is close to the centre of the cage. At this curvature the bat is unable to generate sufficient vertical force to support the weight in a level turn in addition to balancing the lateral acceleration: even though it reduces speed considerably to reduce the lateral acceleration and to increase curvature, the animal must still tolerate some loss of height. However, for the short period of the turn the height loss is relatively small, and the bat can offset it by decelerating and using kinetic energy to gain height during the approach. Surprisingly, maximum elevation is reached at the start of the turn rather than at the middle: potential energy is used to maintain and increase speed during the turn in order to offset the loss of thrust from the wings, since lift must be primarily responsible for the normal force during this phase; slight deceleration parallel to the flight path can be distinguished at this stage. In this example excess height is probably lost deliberately after the turn.

A detailed discussion of turning mechanics and a comparative analysis of turning by different species will be presented elsewhere. Similar factors constrain turning performance in other microchiropteran bats: in slow, tight, powered turns, animals are limited by their inability to generate sufficient vertical and lateral force simultaneously. In this way aerodynamic, mechanical and sensory factors interact to set the maximum turning speed and the minimum radius within which an animal can successfully turn.

Accurate reconstruction of the flight path and of body movements in three dimensions is essential if the mechanical factors underlying flight performance and manoeuvrability are to be understood. The example of a slow, powered turn in a small bat shows the importance of determining precisely acceleration and flight path geometry in three dimensions in order to explain how the animal is able to turn in a space smaller than its wingspan without overall loss of mechanical energy. The resection method described here is particularly appropriate for such experiments, and has proved extremely robust and tolerant of inaccuracies in the experimental configuration. By employing a test cube with minimum geometrical constraints, it is possible to estimate the camera location and magnification sufficiently accurately to reconstruct a complex object in three dimensions with a single microcomputer. With the added use of a stroboscope the method may be used also to determine velocity, acceleration and curvature of the flight path.

This research was funded by the Science and Engineering Research Council, the Natural Environment Research Council and the Royal Society. The authors wish to thank Dr Geoff Spedding for advice with photogrammetry, and an anonymous referee for some helpful comments.