python - How to find the center point of this rectangle

Question

I am trying to find the center point of the green rectangle which is behind the fish, but my approach is not working. Here is my code:

#Finding contours (almost always finds those 2 retangles + some noise):
_, conts, hierarchy = cv2.findContours(img_green, cv2.RETR_TREE , cv2.CHAIN_APPROX_SIMPLE)
for cnt in conts:
    area = cv2.contourArea(cnt)

    #filter noise
    if area > 25:

        M = cv2.moments(cnt)
        x1, y1, w, h = cv2.boundingRect(cnt)
        x2 = x1 + w                           # (x1, y1) = top-left vertex
        y2 = y1 + h                           # (x2, y2) = bottom-right vertex
        cy = int(M['m01']/M['m00'])           # (cx, cy) = rect center
        cx = int(M['m10']/M['m00'])
        rect = cv2.rectangle(green_bar_win, (x1, y1), (x2, y2), (255,0,0), 2)
        center = cv2.circle(green_bar_win, (cx, cy), 2, (0,0,255), 4)

As you can see, it finds the rectangle's contour but divided where the fish is, making 2 different shapes. It also finds the center of this 2 shapes (the blue points), however I don't know how to find the middle of the big one. I thought about averaging all the found rectangle centers but I don't know how to write this out. I am finding the rectangles by hsv color. Help?

EDIT: I have 'y1' from the top rectangle, but don't know how to get y2 from the bottom one while inside the for loop. I tried this:

_, conts, hierarchy = cv2.findContours(img_green, cv2.RETR_TREE , cv2.CHAIN_APPROX_SIMPLE)
for cnt in conts:
    area = cv2.contourArea(cnt)

    #filter noise
    if area > 25:

        x1, y1, w, h = cv2.boundingRect(cnt)
        x2 = x1 + w                           # (x1, y1) = top-left vertex

        try:
            rect_center = np.average([y1, y2])
        except:
            print("Failed to average")

        y2 = y1 + h                           # (x2, y2) = bottom-right vertex
        rect = cv2.rectangle(green_bar_win, (x1, y1), (x2, y2), (255,0,0), 2)

But it still fails, because y2 is used before assigment. So, how could I get y2 from the second loop itineration before y1 is overwritten by the 'for' loop?

Answer 1

It's time to break this to a full answer.

After you identify each rectangle, compare its corners to each existing shape. If it has the same color and a pair of corners in common (shared edge), then update the old rectangle: replace those two corners with the other corners of the new rectangle.

Add a couple of things to your code:

1. keep a list of the shapes found so far

2. as you find each shape, compare to the ones found already.

   rect_list = []

   for cnt in conts: area = cv2.contourArea(cnt)

   #filter noise
   if area > 25:
   
       M = cv2.moments(cnt)
       # ... several lines deleted
       rect = cv2.rectangle(green_bar_win, (x1, y1), (x2, y2), (255,0,0), 2)
       center = cv2.circle(green_bar_win, (cx, cy), 2, (0,0,255), 4)
   
       # Look for adjacent shapes here
       for old_rect in rect_list:
           # fetch the corners of old_rect
           # if old_rect has a pair of corners in common with
           #     the current rect, then merge the two.
           #     Do this by expanding the corners of old_rect.
           else:
               rect_list.append(rect)