Thermal imaging of stress: you won’t believe your eyes

The following is a guest blog by Dr. Joshua Robertson Tabh


In my short research career, I’ve come to accept (even relish) that there are some projects that endlessly surprise; projects with shifting objectives that find you running drive-by thermal camera hand-offs along the QEW at questionable hours. The project that I’m about to describe is one of “those”. And curiously, despite the innumerable twists and turns, it just so happened to be a project with some of the most useful outcomes I’ve helped to produce. In this guest post, I’ll describe those outcomes.

But first, let’s begin in 2016. I had just begun my PhD research in avian stress physiology, and mere months before, Paul Jerem and others had released a highly intriguing protocol which suggested that the physiological stress response could be detected, and possibly quantified, in birds by simply measuring changes in body surface temperature (https://www.jove.com/t/53184/thermal-imaging-to-study-stress-non-invasively-in-unrestrained-birds). The rationale behind their protocol was that following exposure to a stressor, the sympathetic nervous system triggers vasocontriction of blood vessels at the skin (among other things), which manifests as measurable changes in skin temperature. This idea isn’t new. Rather, it likely dates back to the early 20th century or previous (e.g. Wolff and Mittelman, 1937). However, Jerem et al’s protocol was the first to show that a stress-induced change in skin temperature could be detected at the eye region in a wild bird, using infrared thermography (see Edgar et al, 2013, for a study in chickens). A clever application of thermography.

Jerem et al’s work was exciting. But a few important questions seemed to linger:

(1) how well does this stress-induced change in eye region temperature reflect circulating changes in sympathetic nervous system markers (i.e. catecholamines, like adrenaline and noradrenaline)?

(2) Are changes in surface temperature at other bodily regions better indicators of the physiological stress response than the eye region (e.g. the bill: https://journals.biologists.com/jeb/article/223/8/jeb220046/223869/Body-surface-temperature-responses-to-food)?

And (3) how robust and reliably detectable are stress-induced changes in body surface temperature? More specifically, how resilient is this response to masking by changes in bird position (see: https://besjournals.onlinelibrary.wiley.com/doi/abs/10.1111/2041-210X.13563)?

So, being nagged by these questions, a team of ecophysiologists (Glenn Tattersall, Gary Burness, and Oliver Wearing), an endocrinologist (Gaby Mastromonaco), and myself sought answers.

To do so, we required an experimental approach that would allow us to measure both body surface temperature (here, at the eye region and bill) and circulating catecholamines in “stressed” and “unstressed” birds. However, measuring circulating catecholamines requires sampling blood. And since puncturing a vein with a syringe is surely sufficient to activate a physiological stress response on its own (thus rendering “unstressed” birds “stressed”) blood sampling by this standard method simply wasn’t possible. Ideally, we would fit a sample of birds with central venous catheters to permit blood sampling without capture and venipuncture.   This approach could work, however, even if blood samples were to be collected effectively, catecholamines can be a pain to quantify, even for contracted labs with high-end machinery. It’s for this latter reason that we accepted the reality of leaving our first research question unanswered. 

Fig. (1) Domestic pigeon being monitored during rest, before experimentation.

Nevertheless, we could persist with a simple experimental design to answer research questions (2) and (3); quite simply, thermographically image birds during rest (Fig 1) and during a stress exposure (for us, handling). To answer question (2), we would then quantify and compare the magnitude of stress-induced changes at the eye region and bill. And lastly, to answer question (3), we would aim to test the effect of head angle on our ability to detect stress-induced changes in eye region and bill temperature. In theory, a nice and clean approach.

Before I get to the answers of our remaining research questions,  a small note on how we estimated head angle (for the interested reader).

Estimating Head Angle from 2D Image

Estimating the orientation of a 3D object from a 2D angle has been a concern for humans since photography was invented. Among mathematicians, this challenge has since acquired a formal name: the “perspective-n-point” (or “PnP”) problem. All solutions to the PnP problem first require knowledge of where, in a 2D plane, at least 3 points in an imaged object lay. We’ll call these points “landmarks”. Of course, more than 3 landmarks are best to improve estimation accuracy, but most agree that 3 will do for a reasonable guess. Next, rough dimensions of the imaged object in 3D space are needed. Such dimensions must be sufficient for one to estimate where the chosen landmarks may lie, relative to each other, in a theoretical 3D co-ordinate system known as the “world co-ordinate system”.

Once this information is collected, several geometrical approaches may be used to calculate how the imaged object must have moved or rotated such that the landmarks in 3D space overlap with those observed in 2D space (after adjusted for lens distortion). Interestingly, there is one industry with considerable investment in creating efficient geometrical approaches: virtual reality (or “VR”) gaming. Why? Because using VR gaming requires that the system can estimate the gamer’s 3D position at all times (with, interestingly, tiny infra-red lights implanted in the headset as landmarks). Thanks to this investment by the VR industry, studies developing and comparing the accuracy of geometrical solutions to the PnP problem are flourishing. It’s a perfect time for biologists like us to start taking a peak at them.

For our study, we chose to use to an approach called the “EPnP” that was first proposed by Lepetit and others in 2009 (https://link.springer.com/content/pdf/10.1007/s11263-008-0152-6.pdf). We chose this approach because it permits one to use >4 landmarks for positional estimation (thus reducing error) with little cost to computational time relative to traditional solutions. Other approaches have been lauded for improving accuracy (e.g. P3P with RANSAC) and we encourage others to pursue those approaches. For our study, however, we were interested in balancing accuracy and efficiency.

To execute the EPnP approach, we estimated the 2D position of up to 9 landmarks on a pigeon’s head by loading our thermographic images into ImageJ (Fig 2). Building a 3D model turned out to be much less time consuming – simply draw on morphometric measurements of domestic pigeons reported in literature. From these data, and EPnP algorthims, we were thus able to estimate both a 3D translation and 3D rotation of an imaged bird’s head, relative to a virtual model of a perpendicularly facing individual.

Fig. (2) Thermographic image of a domestic pigeon with black dots marking 6 of 9 possible landmarks. The black line at the tip of the bill indicates the estimated direction in which the pigeon is facing.

Our results?

I’ll break them down by question.

Question (1): How well does this stress-induced change in eye region temperature reflect circulating changes in sympathetic nervous system markers (i.e. catecholamines, like adrenaline and noradrenaline)?

Answer: Yet unanswered.

Question (2): Are changes in surface temperature at other bodily regions better indicators of the physiological stress response than the eye region (i.e. the bill: https://journals.biologists.com/jeb/article/223/8/jeb220046/223869/Body-surface-temperature-responses-to-food)?

Answer: Our results suggest that, at least in our study species, surface temperature the bill is probably a better indicator of stress physiological state. I’ll explain why by referencing what we observed from data that did not control for head position. After stress exposure, bill temperature fell significantly by ~4°C after stress exposure (handling), while eye region temperature did not significantly change (Fig 3). Rather, temporal patterns in eye region temperature appeared remarkably similar between “stressed” and “unstressed” birds. Moreover, only stress-induced changes in bill temperature showed significant inter-individual variation, suggesting that if one wishes to build a metric of “stress-responsiveness” from changes in surface temperature, doing so at the bill is likely more effective than at the eye region.

Fig. (3) Changes in eye region and bill temperature across time in both stress-exposed and control birds. Time 0 (marked with a vertical dashed line) indicated that time that flight cages were opened to permit capture and handling of birds in the stress-exposed treatment group. Dots represent averages across birds per 5 seconds of observation, and lines of best fit represent trends estimated by generalised additive mixed-effects models. Ribbons represent 95% confidence intervals around trend estimates.

Question (3): How robust and reliably detectable are stress-induced changes in body surface temperature?

Answer: It depends on where you look. After correcting for changes in head position in our birds, a significant effect of stress-exposure on eye region temperature emerged (Fig 4). This was not the case for stress-induced changes in bill temperature, which were detectable regardless of whether head position was accounted for or not. This point, we think, is particularly important for two reasons:

(1) don’t correct for changes in object position and you risk missing out on detecting biological processes, and

(2) surface temperatures of some body regions might be better indicators of your biological process of interest than others.

Fig. (4) Change in eye region temperature of stress-exposed and control pigeons after correcting for changes in head orientation. Again, time 0 (marked with a vertical dashed line) indicated that time that flight cages were opened to permit capture and handling of birds in the stress-exposed treatment group. Dots represent averages across birds per 5 seconds of observation, and lines of best fit represent trends estimated by generalised additive mixed-effects models. Ribbons represent 95% confidence intervals around trend estimates.

Take Home Message

To conclude, drawing biological inference from thermographic images is tricky. Many sources of error can get in the way of your ability to meaningfully do so, and a common one is changes in object position. As such, biologist should always remember to correct for object position when working with their surface temperature data – perhaps by using our method or another. 

Read the full study here: https://physoc.onlinelibrary.wiley.com/doi/10.14814/phy2.14865

Citation

Tabh, Joshua KR, Burness, G, Wearing, OH, Tattersall, GJ, Mastromonaco, GF.  2021. Infra-red thermography as a technique to measure physiological stress in birds: body region and image angle matterPhysiological Reports, Accepted. https://doi.org/10.14814/phy2.14865

Acknowledgements

Dr. Joshua Robertson Tabh is a graduate of Trent University, co-supervised by Dr. Gary Burness and Dr. Gaby Mastromonaco. This research was made possible with the cooperation of the Toronto Zoo and by the watchful eye of Oliver Wearing. Since 2016, Joshua and Glenn have shared many conversations about avian physiology, imaging, and coding and Glenn invited Joshua to guest author this post after all these efforts finally reached the publication stage.

Funding for this research was provided by the Toronto Zoo Foundation, an NSERC Collaborative Research and Training Experience Program (Grant #: CREATE 481954-2016), a Howard P. Whidden grant to OHW, and an NSERC Discovery Grant to GJT (Grant # RGPIN-2014-05814).

Lend us your thermal images!

Please consider taking part of an open repository initiative of thermal images hosted at the following website: https://trench-ir.azurewebsites.net/. If you are acquiring thermal images of plants, animals, or their environment using FLIR cameras, we would like you to share your images as part of this initiative. We welcome images from research grade cameras or from hand-held mobile phone provided the images are radiometric jpgs.

Infrared imagery offers a unique opportunity to see biophysical properties in real time. We can watch organisms heat up, cool down, and generally transfer heat back and forth throughout their environment. In the TrEnCh-IR Project, we use infrared imagery to help people see the world from a thermal perspective because we believe it’s an intuitive first step to understanding microclimate and the impacts of warming.

The TrEnCh-IR project is part of a larger initiative interested in Translating Environmental Change into organismal responses. Our goal is to build case studies of how animals are impacted by climate change to improve our approach to climate change biology education, policy, and research.

Mission

FLIR cameras are extensively used, increasingly so with the availability of FLIR thermal cameras that attach to phones. However, the cameras produce images in a non-standard format (radiometric jpgs) and analyzing the images requires purchasing expensive FLIR software. Project collaborator Tattersall has produced an open source R package (ThermImage, https://github.com/gtatters/Thermimage) that converts the images into standard formats and extracts additional data to allow analysis in commonly used and open source software such as ImageJ. Our web service makes these tools more accessible. We aim to empower more people to view the world from a thermal perspective.

Our thermal image repository will allow researchers to analyze the surface temperatures of disparate organisms in diverse environments. Education and outreach resources promote understanding how organisms experience their environment. We aim to maintain the repository long term, but can not guarantee longevity at this point. An ongoing aim is to use initial AI algorithms, potentially combined with crowd sourced landmarking, to distinguish organisms, particular body parts, and backgrounds. Our interface will allow users to explore the images to understand how organisms interact with their environments.

Motivation

Currently most analyses of the impacts of climate change on organisms are based on air temperatures, but body temperatures of ectotherms can differ from air temperatures by tens of degrees. Additionally, the characteristics and behaviours of organisms can result in their experiencing different body temperatures even in the same environment with repercussions for species interactions. Moving beyond air temperatures to consider body and surface temperatures may thus be essential to accurately forecasting climate change impacts. Thermal images provide compelling visual examples of why we need to move beyond air temperatures in examining climate change impacts as well as data that can inform approaches for modelling how organisms interact with their environment.

Team

Dr. Lauren Buckley, University of Washington, Professor

Abigail Meyer, Lead Developer & University of Washington Research Scientist

Dr. Glenn Tattersall, Brock University Professor & Thermal Biologist

Site development based with University of Washington, in the Department of Biology.

We are funded by AI for Earth, a Microsoft initiative.

Toucans of the atlantic

Earlier this summer, I was lucky enough to visit the Isle of May, Scotland to fulfill a long-time ambition to collect thermal image data on puffins in the wild. Ever since we published our work on the toucan in 2009, I have wanted to study the puffins, examining evidence for elevated capacity to control or distribute body heat through their uniquely colourful bill. Living in a cool climate with a large radiator like their bill presents a unique opportunity to test our hypotheses. In spring of 2018 I managed to visit the Elliston, Newfoundland puffin colony to start this project, but the distance to view a little too far to obtain high quality results.

Well, the short story is that they do show an extraordinary capacity to do so! Here is just a sample image (from the 200 Gb of videos):

Active and basking Atlantic puffins show capacity for intense heat transfer to the bill. The one above has recently landed back at the colony, presumably foraging although in this case, there is no evidence of food. Other images show cool bills, as we have seen in many other bird species, demonstrating the vasomotor control over blood flow to the bill is a fairly generalised phenomenon.
Infrared thermal video of an Atlantic puffin in May 2018 – early arrival at nest and investigating burrows.
Atlantic puffin in the rain.

If I only had the time to conduct the data analysis, I could put some numbers on these values. I certainly have my work cut out for me, examining those returning from the water with food vs. those basking and resting. I have a few other thoughts about these data that I hope to extract.

Many thanks must go to the town of Elliston, Newfoundland and the Atlantic puffin colony there, the Centre for Ecology and Hydrology (UK), the Isle of May (Scotland) Scientists, and especially Mark Newell for hosting me at the Isle of May, and Mike Harris for introducing us. Sorry it took so long to post this.

Further Reading

Tattersall, GJ, Arnaout, B, and Symonds, MRE.  2017.  The evolution of the avian bill as a thermoregulatory organ. Biological Reviews 92: 1630-1656. doi:10.1111/brv.12299

Greenberg, R, Cadena, V, Danner, RM, and Tattersall GJ. 2012. Heat loss may explain bill size differences between birds occupying different habitats. PLoS One, 7: e40933. 

Symonds, MRE and Tattersall, GJ. 2010. Geographical variation in bill size across bird species provides evidence for Allen’s rule.American Naturalist. 176: 188-197.

Tattersall, GJ, Andrade, DV, and Abe, AS. 2009. Heat exchange from the toucan bill reveals a controllable vascular thermal radiator.Science, 325: 468-470.

Delayed homeothermy in high-atltitude deer mice

Congratulations to Cayleih Robertson for her paper that just came out today in the Proceedings of the Royal Society B, entitled:

Development of homeothermic endothermy is delayed in high-altitude native deer mice (Peromyscus maniculatus)

This project is the result of Cayleih’s PhD research in Dr. Grant McClelland’s lab. I was lucky enough to be involved with these talented scientists, although all the credit goes to Cayleih for her hard work. What a tour de force of physiology, biochemistry and imaging.

Sample video of the experimental paradigm examining cooling in neonatal mice using infrared thermal imaging. For more details, see the link to the paper above.

Insect thermogenesis

A luna moth was on our window outside the department yesterday.

So, we brought it inside for a lab show and tell. Shivering up a storm….

Thorax temperature up to 32C, while the abdomen temperature was still below 20C (the lab was ~21C, and the moth had been briefly placed in a cold fridge).

Shape Shifting Birds – PhD Opportunity

Please consider applying for this PhD Opportunity in Australia to work with my colleague, Dr Matthew Symonds on Shape-Shifting Birds.

This research forms part of an ARC Discovery Project (PI: Symonds; CI: Klassen & Tattersall) whose goal is to determine whether changes in body shape are an evolutionary response to climate change. Endothermic animals (such as birds) have a range of adaptations for dealing with the temperatures they experience. One such adaptation is body shape: birds in warmer climates tend to have large extremities (bills and legs), increasing their surface area and enabling loss of excess heat. Adaptations to climate (and hence climate change) can occur quickly, and there is evidence of significant increases in bird extremities in recent years – a novel potential consequence of climate change. Whether this represents an evolutionary response to climate change is unknown, nor do we know what characteristics make specific bird species liable to respond to climate change in this way, or what the likely consequences of such responses are.

The student will undertake an extensive comparative analysis of Australian birds, designed to identify a) which bird species are showing changes in body shape (bill and leg morphology); b) what ecological (life- history, behaviour, habitat) factors determine such responses; c) whether these changes relate to fitness/survival and d) whether such changes are linked to long-term populations trends in Australian birds.

The project will involve extensive work in Australian museum collections, measuring bird morphology using traditional and modern (3D-scanning) techniques. There is also a strong analytical component, involving use of long-term field data on Australian bird species as well as phylogenetic comparative analysis of large-scale ecological data sets for Australian birds.

Please send an application letter, together with your CV, to Dr Matthew Symonds (matthew.symonds@deakin.edu.au).

Further information can be found in our review papers:

Symonds, MRE and Tattersall, GJ. 2010. Geographical variation in bill size across bird species provides evidence for Allen’s rule.American Naturalist. 176: 188-197.

Tattersall, GJ, Arnaout, B, and Symonds, MRE.  2017.  The evolution of the avian bill as a thermoregulatory organ. Biological Reviews 92: 1630-1656. doi:10.1111/brv.12299

Poetry of Cassowary Casques

So, we published a paper on Cassowary Casques the other day, and then we heard from a Science Communications person that he had written a poem inspired by our research!

http://thepoetryofscience.scienceblog.com/760/the-curious-case-of-the-cassowary-casque

How cool is that!?

The study itself was conducted by Danielle Eastick of La Trobe University (Dr. Kylie Robert and Dr. John Lesku), and published in Scientific Reports recently. Here is a link to the paper.

I won’t link to all the overhyped media reports since they tend to misinterpret (e.g., no, we have not discovered the secret to the casque) the science just like they misinterpreted our toucan bill study.

But here are the main results:

One of the most beautiful and dangerous animals around!

Learning from Tortoises

Our paper on red-footed tortoise reversal learning is now in press! This study represents the efforts of Justin Bridgeman during his honours thesis examining behaviour flexibility and cognition in tortoises. Here is a link to the paper:

https://rdcu.be/bjHU7

Bridgeman, JM and Tattersall, GJ. 2019. Tortoises develop and overcome position biases in a reversal learning task. Animal Cognition. (): 1-11. 10.1007/s10071-019-01243-8

Many thanks to Dr. Miriam Richards, TAs, Tom Eles, and all the students from our animal behaviour course (2013 – 2015) who helped with all the pre-training and Y-maze familiarisation trials that pre-dated Justin’s honours research. And many thanks to all the tortoises who participated.

Tortoise approaching the stimulus (mock experiment with cell phone video)

Tortoise receiving a reward for approaching the correct stimulus.

Here are some sample videos from the supplementary material:

Tortoise in the Y-maze examines both stimuli and slowly approaches the rewarded stimulus on the left.

Tortoise late in the training approaches the rewarded stimulus without pause.

Tortoise moves according to its developed position bias, almost makes an error but corrects itself, and approaches the positive stimulus receiving the reward.

Self-reflection puts fish in hot water

In a narcissistic age, it is not uncommon to think of situations in which spending too long focussed on ourselves can lead to harm.  But what about fish, you might ask?

Today our paper entitled “Social cues can push amphibious fish to their thermal limits” was published in Biology Letters.

This manuscript represents an experimental field study performed in Belize this past spring examining how social information can delay the behavioural thresholds that an amphibious fish exhibits to escape from a thermally stressful environment.  As natural environments are complex systems including abiotic and biotic stressors that may act synergistically, it is important to understand how these interactions operate and impact an animal’s thermal limits.

The mangrove rivulus (Kryptolebias marmoratus), in particular, is an intriguing fish since it routinely leaves the water (emerge) to avoid stressful conditions and in some instances can survive for weeks out of water.  If the water gets too hot or too hypoxic, rivulus will simply jump out and chill out on land.

picture1.png

K. marmoratus contemplating life on land.  Not quite sure yet.  Photo courtesy Keri Martin, Mount Allison University.

img_5007.jpg

K. marmoratus out of water – Image courtesy of Keri Martin, Mount Allison University.

In the present study, we demonstrate that the decision to emerge from water onto land is also socially sensitive.

So, what does this have to do with self-image?  Studying social cues from conspecifics can be a challenge if the conspecifics jump out of water first, so we designed an experiment to provide continuous social cues, using a simple mirror placed underwater.

Tank Schematic.jpg

Supplementary Figure from manuscript

Key Finding: The presence of cues from a conspecific (produced from a mirror) caused the rivulus to delay leaving water until they reach a higher temperature.  This delay is clearly a behavioural decision and not due to enhanced tolerance of warmer temperatures, since their CTmax was not sensitive to the presence of social cuing from the mirror.

This discovery is important since it demonstrates that critical behavioural decisions may affect survival in an animal living in a thermally stressful environment.  Much research into thermal stress to date has focussed on individual responses but social cues are likely just as important to animals in the wild.  We have only addressed what simple visual cues produced by a mirror do to their emersion behaviour.  Extrapolating to how multiple cues will work is, of course, open for future investigation.


 

Full Citation:

Currie, S and Tattersall, GJ. 2018. Social cues can push amphibious fish to their thermal limits.  Biology LettersLink here

 


 

Further Information

K. marmoratus inhabit stress aquatic habitats and can be found often inside crab burrows.

See Dr. Andy Turko’s video from 2012.  It is fascinating to watch:

 

On this year’s trip we took more images attempting to visualise the thermal environments inhabited by the various fish living in the Long Caye.  Here is a thermal image of a crab burrow:

IR_2018-04-23_

Thermal image of a crab burrow (central triangular region) capturing a crab leaving the burrow (bottom centre of image).  Rivulus can be found cohabiting with crabs or each other.  Image taken in April 2018 with a FLIR T1030K thermal imaging camera.

Ever hopeful to observe rivulus emerging spontaneously in the field we set up various camera traps and go pros and time lapse images.

The following video depicts a time lapse video (each frame 10 apart) captured with a simple IR webcam of a outdoor, artificial burrow with water, following 3 rivulus over time.  You can see one emerging in the bottom part of the screen and staying out of water.

 

We want to acknowledge Team Rivulus and Fellow Scientists:

Dr. Patricia Wright, University of Guelph

Dr. Suzie Currie, Acadia University

Dr. Andy Turko, University of Guelph

Dr. Tamzin Blewett, University of Alberta

Dr. Emily Standen

Giulia Rossi, University of Guelph

Louise Tunnah, University of Guelph

Keri Martin, Mount Allison University

Dr. D. Scott Taylor (mangrove.org)

Itza Lodge, Long Caye, Belize