Software such as Mission Planner and Matlabare used in the laboratory setup. Mission is a ground control station.Mission Planner can be used as a configuration utility or as a dynamic control supplement for your autonomous vehicle. Matlab is used for image processing. Image from camera is processed in Matlab and position of target in image is computed. By means of which PWM signal required for bringing the target in centre of image is calculated. Controlled signal is sent to servo only by mission planner. Thus, to send continuous signal from Matlab to mission planner, virtual socketing is made.Mission planner has an option for python scripting by which we can control the MAV through scripting instead of giving inputs in GUI.Python scripting option in Mission Planner is used to create socket. In Matlab virtual socket is made forever and control signal is send forever. Missionplanner read the signal from socket and gives input signal to the mentioned channel. By configuring the input and output pins for gimbal servo, control signal from Matlab is able to control the gimbal servo.
IV. CONTROL ALGORITHM
For controlling of camera, object has to be tracked continuously.For tracking of target in image, Mean shift algorithm with background histogram is used. Position of target in image is computed by Mean shift algorithm. Deviation of the position of target in image from the center of image is calculated. According to the deviation, controlled input to servo is computed by the algorithm. Since we use 2-axis gimbal, pan motor and tilt motor corresponds to x and y coordinates position of target in image respectively. Let (xc,yc) be the center of the image, computed from image height and width of image. xd,yd be the computed position of target in image.
Fig4 shows control scheme for control of gimbal. According to error control algorithm will compute PWM signal to rotate the servo.PWM generation for servo control depends up on the properties of servo used in gimbal and alignment of servo arm with the camera
4.1. Properties of Servo
Angle Range: ± 45°
(increase or decrease above PWM range won’t have effect in movement of servoarm)
For maximum angle change 90⁰, PWM change of 800s is required. Thus for 1⁰ angle rotationPWM change of 8.89s is required. For PWM change from
servo arm rotates from -45⁰ to 0⁰ and for PWMchange from 1500 to 1900 servo arm rotates from 0⁰ to +45⁰. x co-ordinate and y co-ordinate of position of target in image is used to control Pan and tilt servo respectively. Open loop response of system is considered to study the movement of camera and its effect in change of position of target in image.
4.2. Open Loop Response of System
Camera produces an image of size 720 480 pixel with 360240 as center of image.When pan PWM is decreased, the target in image moves right in image. When pan PWM is increased, the target in image moves left in image. When tilt PWM is decreased, the target in image goes up in image. When tilt PWM is increased, the target in image goes down in image. From these inferences, to control the camera, following points is must.
1. When target is in left half of image (in real target will be in right side of camera FOV), to bring target in center of image, PWM given to pan servo should be decreased.
2. When target is in right half of image (in real target will be in left side of camera FOV), to bring target in center of image, PWM given to pan servo should be increased.
3. When target is in upper half of image (in real target is below the camera FOV), to bring target in center of image, PWM given to tilt servo should be increased.
4. When target is in lower half of image (in real target is above the camera FOV), to bring target in center of image, PWM given to tilt servo should be decreased.
In order to find relation between pixel difference and PWM required for having the target in center, Open loop response of system for the target placed 5 feet away from the camera is considered. Initially servo PWM for pan is at 1500s and servo PWM for tilt is at 1100s.At this position of camera, the target is at (361,241) in image (i.e. approx. center of image). Now, tilt servo PWM is kept constant and pan servo PWM is changed sequentially.
When servo PWM is changes and its corresponding pixel shift in position of target in image is calculated. Table-1 shows the target position shift in image for change in pan servo PWM from its initial value while tilt servo is kept constant. There is no considerable change in y-coordinate of position of target in image. Fig-5shows the pixel change in position of target in image with respect to the initial position of target in image. When PWM is increases more than 200s targets goes out of image. It inferences at a distance of around 5 feet, 1 pixel change requires 0.7692s PWM change. So for dx pixel, (0.7692 dx)s PWM change is required. But in real time when servo suddenly moves large, image processing algorithm cant able to track the target, so rate of change of PWM is decreased. Rate of change is acquired from experiments.
If target is in right half of image pixel difference dx computed will be in negative and vice-versa. From the inference of all these experiments, the empirical formula used for gimbal camera pan servo control to have target in center is
Closed loop experiments are carried out with empirical equation, if we add the control equation in the APM 2.5 source code it will takes more time for processing. In order to overcome this problem, PWM is computed in Matlab. Through virtual socket, Mission Planner able to read the PWM values from Matlab and given as input to the APM 2.5 board.
5.1. Stationary Object
Consider a stationary target, which is kept 5ft away from camera in such a way that when pan servo is at PWM 1200s and tilt servo is at PWM 1100s, target is at corner of image. (i.e. maximum distance from center of image).Depending upon difference in pixels from center of image, algorithm compute PWM is shown in Table-2. Change of position of target in image according to servo arm rotation is shown in Fig.6. Algorithm took 0.856 secto bring the target from corner of image to center of image.
5.2. Moving Object
Initial position of servo arm should be given for algorithm to track the target, because PWM is changed consecutively according to pixel difference in image. If the manual input of Initial position of servo arm is different from the actual position of arm, then in succeeding frame algorithm will lose the target from frame. Position of servo arm (i.e initial PWM) displayed in the Mission Planner is manually entered in the algorithm. The target selected by the operator in first frame is given as input to the algorithm, and then algorithm starts moving the camera to have target in center of image. Laboratory experiment is conducted by moving target randomly with camera at fixed place. Since camera is kept constant, target is made to move within the FOV of camera.
Target is moved randomly with some boundary condition. (i.e. target should be always in image frame)
Initial position PWM of servo arm should be given correctly.
Fig7 shows the pixel difference dx and its corresponding PWM generated for pan servo by the algorithm.
Since it is feedback control, control action is taken after the effect. Thus, when target suddenly changes its path, error is large at that frame, then algorithm starts decreasing it
When error is large, the controller output is also large and vice-versa.When the error is around zero, PWM is constant at that point.