Machine vision systems protect endangered birds

On account of the increasingly strict wildlife conservation regulations, wind turbines have to be switched off when a protected species of bird approaches them.

With the help of artificial intelligence and components from STEMMER IMAGING, a machine vision system from phil-vision banishes the danger of birds being struck by wind turbine blades and minimises expensive downtimes.

The wildlife conservation regulations for the construction and operation of wind turbines have been tightened enormously in the past years. For example, if bird species that are sensitive to wind power are present within a certain radius of wind turbines, the turbines generally have to be switched off for several days as soon as fieldwork such as ploughing, threshing or mowing takes place. During these periods the birds are at increased risk as they spend more time over these fields in the search for food.

Also, when protected birds nest close to existing turbines, these turbines may only be operated with restrictions. Therefore, whole-day shut-downs are often included in permits during the breeding period from March to August.

However, every day on which a wind turbine doesn't produce electricity reduces its cost-effectiveness. Hence, despite the political will to expand the use of renewable energies compared to fossil fuels, the number of wind turbines being built has reduced sharply, in particular, due to the wildlife conservation regulations.

With the help of machine vision systems and artificial intelligence, this problem can now be solved in such a way that both the necessary cost-effectiveness for the wind farm operator and the needs of wildlife conservation are accounted for.

In order to avoid long, expensive shut-down times in which there are often no animals in danger, Bürgerwindpark Hohenlohe GmbH is currently developing a system with the support of phil-vision GmbH with which large birds of prey can be recognised and localised with their flight paths tracked. The aim is to shut down wind turbines only when protected birds move within a certain distance of the turbines.

Demanding task

According to phil-vision founder Gregor Philipiak, the recognition of birds and their classification for the subsequent decision as to whether the bird is a protected species is an extremely demanding task: "The fauna differs in every location of a wind turbine, so it's necessary to create individual protection concepts. With the first monitoring system, we initially elaborate on the specific requirements so that we can then design the target system to suit the individual turbine."

According to Philipiak the results so far have been very promising: "At present, we have about ten installed test systems." Such a system consists of six industrial colour cameras with a resolution of 6 or 20 megapixels, which are fixed to the mast of a wind turbine in weatherproof protective enclosures and capture images to monitor the complete airspace around the wind turbine."

To capture the entire surrounding area with a 360° all-round view, special wide-angle lenses were used so large fields of view can be surveyed.

Hundreds of thousands of images

The evaluation of the captured images and the reliable recognition of endangered birds represent the actual skill in this application according to Philipiak: "Our system works on the basis of deep learning methods and for the learning phase it requires almost 400,000 images of birds of different types, at different distances and with various flight positions. Added to that there are about a further 100,000 images (per negative instance) such as insects, aircraft or helicopters as well as the captured images with the local special features."

From this large number of training images a powerful development system automatically creates a decision tree as well as a classifier, which serves as the basis for the differentiation in the subsequent usage phase.

"Through the use of deep learning methods we are thus creating an intelligent system that automatically detects the animals concerned against the most diverse backgrounds and under varying conditions and distinguishes them from possibly similar objects such as aircraft or flies", Philipiak explains.

For optimisation, the phil-vision system operates in two steps with different resolutions: First, all moving objects are detected with the help of 6-megapixel cameras. By switching to cameras with a 20-megapixel resolution, such objects can subsequently be identified with greater accuracy. The system thus determines whether a detected object is a bird at all and subsequently whether the animal belongs to a protected species.

The program tracks the flight paths of protected birds until they are no longer recognisable and thus no longer represent a potential threat to the operation of the wind turbine. If a protected bird gets closer to the wind turbine than the specified minimum distance, a corresponding signal is sent to the turbine's controller so that it can decelerate the turbine in good time.

For economic reasons, one of the priorities of a wind farm is to have as few false triggerings as possible. Through the use of powerful deep learning algorithms we have been able to fulfil this task better and better over the course of the programming and considerably reduce the number of false shut-downs.

Ideas for the future

In its current version, the system has not yet reached the end of its possibilities, because the development team at phil-vision has further plans. One option for the future is the additional integration of a distance measurement, which is already being tested. This uses two cameras in stereo operation, as a result of which the desired depth information can be obtained.

Prior calibration of all cameras ensures that manufacturing tolerances are eliminated and that the distance determined is correctly converted into metric data. This calibration is implemented with the aid of a drone, which flies to precisely defined points at different heights and distances.

Via a marker, the image processing can subsequently detect the position of the drone in the captured image and store the defined distances for each position. The placing of the respective camera is determined so that precise data for approaching birds can be supplied as a result.

A further planned option is the distinction between the individual protected bird species by means of classification. Deep learning algorithms trained with large quantities of images of different birds are also used for this.

The creation of a database is planned for further evaluation. Collected values for bird occurrence at various locations are to be documented here. A web interface is to offer the possibility to retrieve and display the prepared data specifically for the respective wind farm or the individual wind turbine.

Experienced machine vision partner

phil-vision procures virtually all of the machine vision components required for this innovative system from STEMMER IMAGING. Above all Philipiak values the large selection of components available as well as the extensive know-how of the machine vision experts:

"On account of the demanding application we had to test the most diverse camera and lens combinations during the development to find the optimum configuration. Our partner gave us outstanding support here. There were also special boundary conditions for the cables, which had to be resistant to UV light. STEMMER IMAGING manufactured them for us to the required specifications, which saved us a great deal of effort."

The PC technology for the systems also comes from STEMMER IMAGING. The software for evaluating the captured images was developed by phil-vision using the CVB Image Manager and the CVB Movie tool from the Common Vision Blox imaging library. The use of CVB and Open CV provides users with a high degree of flexibility so that, for example, other cameras can be used if necessary.

Wildlife conservation implemented cost-effectively

It seems successful deployment is not very far: In the first step a finished system for monitoring flight movements of protected bird species in the vicinity of wind turbines is to be used from spring 2020.

In close collaboration with several biologists, protection concepts will then be drawn up on the basis of the respective (flight) behaviour of the birds, in accordance with which the machine vision systems will subsequently be designed and adapted to the specific boundary conditions. The goal should be to meet the wildlife conservation requirements and thus to be able to carry out the urgently expansion of renewable energies in the field of wind power.

www.stemmer-imaging.com

  Ask For More Information…