Python kalman filter opencv

Python kalman filter opencv

The class implements a standard Kalman filter http://en.wikipedia.org/wiki/Kalman_filter, [237] . However, you can modify transitionMatrix, controlMatrix, and measurementMatrix to get an extended Kalman filter functionality.

Note In C API when CvKalman* kalmanFilter structure is not needed anymore, it should be released with cvReleaseKalman(&kalmanFilter) Examples: samples/cpp/kalman.cpp.

Constructor & Destructor Documentation

◆ KalmanFilter() [1/2]

◆ KalmanFilter() [2/2]

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

Parameters

dynamParams	Dimensionality of the state.
measureParams	Dimensionality of the measurement.
controlParams	Dimensionality of the control vector.
type	Type of the created matrices that should be CV_32F or CV_64F.

Member Function Documentation

◆ correct()

Updates the predicted state from the measurement.

Parameters

measurement

The measured system parameters

Examples: samples/cpp/kalman.cpp.

◆ init()

Re-initializes Kalman filter. The previous content is destroyed.

Parameters

dynamParams	Dimensionality of the state.
measureParams	Dimensionality of the measurement.
controlParams	Dimensionality of the control vector.
type	Type of the created matrices that should be CV_32F or CV_64F.

◆ predict()

Computes a predicted state.

Parameters

control

The optional input control

Examples: samples/cpp/kalman.cpp.

Member Data Documentation

◆ controlMatrix

control matrix (B) (not used if there is no control)

◆ errorCovPost

posteriori error estimate covariance matrix (P(k)): P(k)=(I-K(k)*H)*P'(k)

◆ errorCovPre

priori error estimate covariance matrix (P'(k)): P'(k)=A*P(k-1)*At + Q)*/

◆ gain

Kalman gain matrix (K(k)): K(k)=P'(k)*Ht*inv(H*P'(k)*Ht+R)

◆ measurementMatrix

◆ measurementNoiseCov

measurement noise covariance matrix (R)

◆ processNoiseCov

process noise covariance matrix (Q)

◆ statePost

corrected state (x(k)): x(k)=x'(k)+K(k)*(z(k)-H*x'(k))

◆ statePre

predicted state (x'(k)): x(k)=A*x(k-1)+B*u(k)

Источник

Kalman filter, predict the trajectory of an Object

Follow the instructions carefully and download the material you need to best follow this tutorial.

1. Kalman filter with dots on solid background

In the downloaded files, you will also find the main.py file already complete but in the tutorial I will explain it step by step and I advise you to write it yourself from scratch to better memorize the procedures.

First of all import kalmanfilter.py and the OpenCV library

from kalmanfilter import KalmanFilter import cv2

Now initialize Kalman filter and we try to insert values to get a prediction

# Kalman Filter kf = KalmanFilter() predicted = kf.predict(50,50) print(predict)

As you can see from the image below, if we print the result, we will get (0,0) because the Kalman filter analysis function needs more values to make a prediction.

More precision with more values

By inserting more position points you will get a more and more precise prediction. Here is an example of code to insert more points that will make our Kalman filter better.

# Kalman Filter kf = KalmanFilter() predicted = kf.predict(50,100) predicted = kf.predict(100,100) predicted = kf.predict(150,100) predicted = kf.predict(200,100) print(predict) # result (238,114)

Simulate the movement of a ball

To verify and put into practice the theory we have seen in the previous paragraph, we simulate the trajectory of a ball that goes from right to left. We can do everything with Opencv by specifying a series of points where the ball will be.

At the same time we pass all points to the function kf.predict(x, y)

from kalmanfilter import KalmanFilter import cv2 # Kalman Filter kf = KalmanFilter() img = cv2.imread("blue_background.webp") ball_positions = [(50, 100), (100, 100), (150, 100), (200, 100), (250, 100), (300, 100), (350, 100), (400, 100), (450, 100)] for pt in ball2_positions: cv2.circle(img, pt, 15, (0, 20, 220), -1) predicted = kf.predict(pt[0], pt[1]) cv2.circle(img, predicted, 15, (20, 220, 0), 4) cv2.imshow("Img", img) cv2.waitKey(0)

By executing the code you can see from the image the solid red balls that go from left to right and the green circles that make the prediction of the next position.

As you can see at the alignment the first green circle is on the point with coordinate (0,0) because it does not have enough values to do the processing. Instead, the last green circle indicates, in an absolutely plausible way, the next possible position of the red ball.

Trying to add more points for prediction with this code

for i in range(10): predicted = kf.predict(predicted[0], predicted[1]) cv2.circle(img, predicted, 15, (20, 220, 0), 4) print(predicted)

you can see how the Kalman filter manages to make a correct prediction. In the image below you can see other green circles that assume the position of the red ball

The same procedure is valid if I simulate the launch of a ball whose trajectory draws an arc. Again Kalman filter improves from time to time as new points are awarded. The one in the image below is the result.

2. Use kalman filter to predict the trajectory of real object

In the previous chapter, we used the Kalman filter to predict a simulated red ball, now we will do it with a real object: an orange from a real video. You can already find everything in the orange_prediction.py file but I recommend that you follow the steps carefully.

Detect the object

The first step consists of object detection, in this case of an orange, identified with the color recognition method. We will not dwell on this aspect because it is not the subject of this tutorial and you just need to include the orange_detector.py file to make it work with the video I am using in the example.

We import everything necessary and proceed with obtaining the frames from the video through a loop

import cv2 from orange_detector import OrangeDetector from kalmanfilter import KalmanFilter cap = cv2.VideoCapture("orange.mp4") # Load detector od = OrangeDetector() # Load Kalman filter to predict the trajectory kf = KalmanFilter() while True: ret, frame = cap.read() if ret is False: break

Now We use the functions for our orange detector and we get the coordinates of 2 points. However, these correspond to the top-left point and the bottom right point of the object. With a small mathematical operation, we obtain the coordinates of the center.

orange_bbox = od.detect(frame) x, y, x2, y2 = orange_bbox cx = int((x + x2) / 2) cy = int((y + y2) / 2)

Now we have what we need, we just need to add the Kalman filter function to predict the future position of the object.

predicted = kf.predict(cx, cy) #cv2.rectangle(frame, (x, y), (x2, y2), (255, 0, 0), 4) cv2.circle(frame, (cx, cy), 20, (0, 0, 255), 4) cv2.circle(frame, (predicted[0], predicted[1]), 20, (255, 0, 0), 4)

Showing a small red circle for the current position and a blue circle for prediction through Kalman filter, this is the result extrapolated from a single frame

3. Kalman filter on real scenarios

This algorithm is very important for object tracking. For example, in the case of overlapping objects with a specific ID, it is possible not to confuse them thanks to the prediction of their displacement.

Kalman filter is also used in more complex tracking algorithms such as SORT and Deep SORT and this allows its use in real scenarios. I have covered these topics in my Object Detection (Opencv & Deep Learning) course where there are complete projects with the use of Deep SORT.

Источник

Kalman Filter Using OpenCV in Python

1. Kalman Filter Using opencv in Python
2. Conclusion

Computer Vision tackles various complicated tasks associated with image and video processing in Artificial Intelligence. We use Python’s opencv library to handle some of these tasks.

This library implements different algorithms and techniques using objects to tackle some of these problems.

One such task is predicting the trajectory of a given object. One of the most common algorithms used for this is the Kalman Filter.

This tutorial will demonstrate the Kalman Filter using opencv in Python.

Kalman Filter Using opencv in Python

The Kalman Filter uses the object’s previous state to predict its next state. This algorithm uses a linear stochastic difference equation to determine the next state.

We need to be familiar with a few matrices associated with this equation.

First, a state transition matrix links the current state to the previous state. Optionally, we can control the input using a control input matrix.

We need to transform the state into some measurement domain which is achieved using a transformation matrix. There also needs to be a process noise vector with covariance.

In Python, we can use the KalmanFilter class from the opencv library to implement this algorithm and predict states. We will define the attributes of an object for this class and assign the necessary matrices.

The measurementMatrix , transitionMatrix , and processNoiseCov attributes specify the previously discussed measurement matrix, transition matrix, and the process noise matrix with covariance, respectively. We can then use the object to make some predictions using the predict() function.

Let us understand this better with an example.

import cv2 import numpy as np  measured=[] predicted=[] dr_frame = np.zeros((400,400,3), np.uint8) mp = np.array((2,1), np.float32) tp = np.zeros((2,1), np.float32)  def on_mouse(k,x,y,s,p):  global mp,measured  mp = np.array([[np.float32(x)],[np.float32(y)]])  measured.append((x,y))  def paint_canvas():  global dr_frame,measured,predicted  for i in range(len(measured)-1): cv2.line(dr_frame,measured[i],measured[i+1],(0,100,0))  for i in range(len(predicted)-1): cv2.line(dr_frame,predicted[i],predicted[i+1],(0,0,200))  def reset_canvas():  global measured,predicted,dr_frame  measured=[]  predicted=[]  dr_frame = np.zeros((400,400,3), np.uint8)  cv2.namedWindow("Sample") cv2.setMouseCallback("Sample",on_mouse); kalman_fil = cv2.KalmanFilter(4,2) kalman_fil.measurementMatrix = np.array([[1,0,0,0],[0,1,0,0]],np.float32) kalman_fil.transitionMatrix = np.array([[1,0,1,0],[0,1,0,1],[0,0,1,0],[0,0,0,1]],np.float32) kalman_fil.processNoiseCov = np.array([[1,0,0,0],[0,1,0,0],[0,0,1,0],[0,0,0,1]],np.float32) * 0.03  while True:  kalman_fil.correct(mp)  tp = kalman_fil.predict()  predicted.append((int(tp[0]),int(tp[1])))  paint_canvas()  cv2.imshow("Output",dr_frame)  k = cv2.waitKey(30) &0xFF  if k == 27: break  if k == 32: reset_canvas() 

In the above example, we implement the Kalman Filter and use it to predict our mouse movement. We create a canvas and move the cursor on this canvas (green color), and simultaneously the Kalman Filter will attempt to predict the cursor movement (red color).

Let us understand what is happening in the code.

We start by creating a canvas frame on which we can draw the cursor movement. The on_mouse() function is used to append the values of the cursor.

The paint_canvas() method takes these values and the predicted values and draws them on the canvas. The setMouseCallback() function is also called whenever the cursor is moved.

We create a KalmanFilter class called the kalman_fil object. The required matrices were assigned using the previously discussed attributes.

Then we run a loop to draw on the canvas and make the predictions.

This class’ correct() method updates the predicted state from the measurement. The predict() function makes the predictions.

These predicted values are given to the paint_canvas() method.

To break out of the loop, we use the break statement, and it is called when the user presses the Esc key (key number 27 on the keyboard). If we press the spacebar, the canvas is cleared of the previous measurement by calling the reset_canvas() method.

Conclusion

To wrap up, we discussed the basics of a Kalman Filter in this tutorial. We discussed the necessary logic and attributes behind it.

We implemented this algorithm using the KalmanFilter class from the opencv library. Different parameters and member functions of this class were demonstrated.

We use the algorithm to predict the cursor’s movement on a drawing canvas.

Manav is a IT Professional who has a lot of experience as a core developer in many live projects. He is an avid learner who enjoys learning new things and sharing his findings whenever possible.

Источник