GIMBALLED CAMERA CONTROL FOR ON-POINT TARGET TRACKING


VPS Naidu*

Citation: NA

Copyright NA


Abstract:

Detailed design of tracking system that enable a Micro Aerial Vehicle (MAV), equipped with a pan-tilt gimbaled camera to be stabilized and to track the target, with target asin center of image. The algorithm performs automatic tracking of target while camera gimbal is moved according to the target position in image.2- Axis gimbal camera set up is made with servo motors and FPV 10x zoom camera.Gimbal camera set up is integrated with APM and XBee to replicate real set up of MAV in laboratory. And it is controlled with GCS set up made of joystick, XBee, Matlab and Mission Planner software. Thus, closed loop for control of gimbal camera designed. Image processing algorithm is used to track the target and to compute the position of target in image. Accordingly, control algorithm will generate control PWM signal for control of camera gimbal.

Description:

I. INTRODUCTION

The past decade has outstanding growth in the use of unmanned air vehicles (UAVs). Even though large UAV are capable of executing complex mission its availability is limited and costly. Thus development of low cost, small UAV (i.e Miniature Aerial Vehicle-MAV) is increased [1, 2]. MAV are used for military and civil application. One of the most significant limitations to small UAV ISR (Intelligent Surveillance and Reconnaissance) systems is their inability to carry a stabilized gimbal capable of delivering the stabilization performance required for high target resolution while the platform stays outside of its detection footprint [3]. In order to have stable gimbal, the target should be locked to the camera. Target locking is the act of maintaining the target in the sensor’s center field of view, under target motion. In order to achieve it, target should be tracked continuously and to have the object in center of image , the camera is moved in Pan and tilt direction according to target position in image. 

II. SYSTEM ARCHITECTURE

Control loop for control of gimbaled camera is shown in Fig-1.In typical operational scenario, when the MAV is in autonomous flight, the system operator may select a target of interest on the screen. Once a target is identified by the operator, the image processing tracking algorithm computes the target tracking box and provide the position of the centroids of this box in every image frame of continuous video from the camera [4]. From the object centroid detected by the algorithm, the required pixel shift to bring the target to center of image is computed. The computed pixel difference is given as the input to an integrated MAV-gimbal control algorithm. Depending up on the pixel error, control algorithm will generate PWM signal for servos of pan ilt unit to keep the target in the center of the image frame [5].  


III. HARDWARE INTEGRATION AND SOFTWARE IMPLEMENTATION
For testing of algorithm, flying MAV is not feasible. So with the hardware and software used for MAV mission, a laboratory set up is made. Following hardware and software are required for making a laboratory set up. 
3.1. Hardware
Hardware component includes AutoPilot Mega (APM 2.5) board, XBee PRO RF transceiver, video transceiver, joystick, servo and camera.
APM 2.5: APM 2.5 is a complete open source autopilot system. It allows the user to turn any fixed, rotary wing or multi rotor vehicle into a fully autonomous vehicle. It will generate PWM signal to run the servo and integrate the system [6].
Xbee Pro Module: The modules operate within the ISM 2.4 GHz frequency band and are pin-for-pin compatible with each other. The XBee PRO RF modules provide quick, robust communication in point to point, peer to peer connectivity with ease. Once it is configured, it communicates with each other. For configuring the module, XCTU software is used [7].
Video Transmitter and Receiver Module: Parton 5.8GHz 1200mW transmitter and receiver kit support the unique needs of low-cost, low-power wireless sensor networks [8]. 
Joystick – Logitech Attack 3 Module: The Logitech Attack 3 is a versatile joystick is used for giving input. The Attack 3 connects to your PC with a wire through a USB Type-A connector [9].
Camera: FPV 10 X Zoom cameras shown in Fig-2 is used. It will produce image in NTSC format and in 700TVL. It has option to adjust focus and zoom. It can zoom till 10x and able to produce high quality image even at 10X. The image produced by this camera is of 720480.
Servo: HS-65 is offered with a choice of shock resistant Metal or zero wear Karbonitegears [10].
Gimbal: 2-axis Gimbal is made with two servo, by means of which it can rotate in pan / tilt direction.Fig-2 shows Gimbaled Camera made with FPV 10X camera and servos in NAL, Bangalore. 

3.2. Software
Software such as Mission Planner and Matlabare used in the laboratory setup. Mission is a ground control station.Mission Planner can be used as a configuration utility or as a dynamic control supplement for your autonomous vehicle[11]. Matlab is used for image processing. Image from camera is processed in Matlab and position of target in image is computed. By means of which PWM signal required for bringing the target in centre of image is calculated. Controlled signal is sent to servo only by mission planner. Thus, to send continuous signal from Matlab to mission planner, virtual socketing is made.Mission planner has an option for python scripting by which we can control the MAV through scripting instead of giving inputs in GUI.Python scripting option in Mission Planner is used to create socket. In Matlab virtual socket is made forever and control signal is send forever. Missionplanner read the signal from socket and gives input signal to the mentioned channel. By configuring the input and output pins for gimbal servo, control signal from Matlab is able to control the gimbal servo.

IV. CONTROL ALGORITHM
For controlling of camera, object has to be tracked continuously.For tracking of target in image, Mean shift algorithm with background histogram is used. Position of target in image is computed by Mean shift algorithm. Deviation of the position of target in image from the center of image is calculated. According to the deviation, controlled input to servo is computed by the algorithm. Since we use 2-axis gimbal, pan motor and tilt motor corresponds to x and y coordinates position of target in image respectively. Let (xc,yc) be the center of the image, computed from image height and width of image. xd,yd be the computed position of target in image.

Fig4 shows control scheme for control of gimbal. According to error control algorithm will compute PWM signal to rotate the servo.PWM generation for servo control depends up on the properties of servo used in gimbal and alignment of servo arm with the camera
4.1. Properties of Servo
Angle Range: ± 45°
PWM range (increase or decrease above PWM range won’t have effect in movement of servoarm)
Neutral position: 
For maximum angle change 90⁰, PWM change of 800s is required. Thus for 1⁰ angle rotationPWM change of 8.89s is required. For PWM change from  servo arm rotates from -45⁰ to 0⁰ and for PWMchange from 1500 to 1900 servo arm rotates from 0⁰ to +45⁰. x co-ordinate and y co-ordinate of position of target in image is used to control Pan and tilt servo respectively. Open loop response of system is considered to study the movement of camera and its effect in change of position of target in image.
4.2. Open Loop Response of System
Camera produces an image of size 720 480 pixel with 360240 as center of image.When pan PWM is decreased, the target in image moves right in image. When pan PWM is increased, the target in image moves left in image. When tilt PWM is decreased, the target in image goes up in image. When tilt PWM is increased, the target in image goes down in image. From these inferences, to control the camera, following points is must.
1. When target is in left half of image (in real target will be in right side of camera FOV), to bring target in center of image, PWM given to pan servo should be decreased.
2. When target is in right half of image (in real target will be in left side of camera FOV), to bring target in center of image, PWM given to pan servo should be increased.
3. When target is in upper half of image (in real target is below the camera FOV), to bring target in center of image, PWM given to tilt servo should be increased.
4. When target is in lower half of image (in real target is above the camera FOV), to bring target in center of image, PWM given to tilt servo should be decreased.
In order to find relation between pixel difference and PWM required for having the target in center, Open loop response of system for the target placed 5 feet away from the camera is considered. Initially servo PWM for pan is at 1500s and servo PWM for tilt is at 1100s.At this position of camera, the target is at (361,241) in image (i.e. approx. center of image). Now, tilt servo PWM is kept constant and pan servo PWM is changed sequentially.
When servo PWM is changes and its corresponding pixel shift in position of target in image is calculated. Table-1 shows the target position shift in image for change in pan servo PWM from its initial value while tilt servo is kept constant. There is no considerable change in y-coordinate of position of target in image. Fig-5shows the pixel change in position of target in image with respect to the initial position of target in image. When PWM is increases more than 200s targets goes out of image. It inferences at a distance of around 5 feet, 1 pixel change requires 0.7692s PWM change. So for dx pixel, (0.7692  dx)s PWM change is required. But in real time when servo suddenly moves large, image processing algorithm cant able to track the target, so rate of change of PWM is decreased. Rate of change is acquired from experiments.
If target is in right half of image pixel difference dx computed will be in negative and vice-versa. From the inference of all these experiments, the empirical formula used for gimbal camera pan servo control to have target in center is 

V. RESULTS
Closed loop experiments are carried out with empirical equation, if we add the control equation in the APM 2.5 source code it will takes more time for processing. In order to overcome this problem, PWM is computed in Matlab. Through virtual socket, Mission Planner able to read the PWM values from Matlab and given as input to the APM 2.5 board.
5.1. Stationary Object
Consider a stationary target, which is kept 5ft away from camera in such a way that when pan servo is at PWM 1200s and tilt servo is at PWM 1100s, target is at corner of image. (i.e. maximum distance from center of image).Depending upon difference in pixels from center of image, algorithm compute PWM is shown in Table-2. Change of position of target in image according to servo arm rotation is shown in Fig.6. Algorithm took 0.856 secto bring the target from corner of image to center of image.


5.2. Moving Object
Initial position of servo arm should be given for algorithm to track the target, because PWM is changed consecutively according to pixel difference in image. If the manual input of Initial position of servo arm is different from the actual position of arm, then in succeeding frame algorithm will lose the target from frame. Position of servo arm (i.e initial PWM) displayed in the Mission Planner is manually entered in the algorithm. The target selected by the operator in first frame is given as input to the algorithm, and then algorithm starts moving the camera to have target in center of image. Laboratory experiment is conducted by moving target randomly with camera at fixed place. Since camera is kept constant, target is made to move within the FOV of camera.

Target is moved randomly with some boundary condition. (i.e. target should be always in image frame)  

Initial position PWM of servo arm should be given correctly.

 Fig7 shows the pixel difference dx and its corresponding PWM generated for pan servo by the algorithm.

 Since it is feedback control, control action is taken after the effect. Thus, when target suddenly changes its path, error is large at that frame, then algorithm starts decreasing it  

When error is large, the controller output is also large and vice-versa.When the error is around zero, PWM is constant at that point.