opengl - Camera and mouse-picking difficulties

Question

Im working on a chess game which is in 3D space. Im trying to figure out how I could move the figures around but camera and different mouse modes are giving me a headache.

Here is an example screenshot:

The Idea:

There are 2 camera/mouse input modes, one allows me to move freely in the space and look around (unlocked camera/fps camera in short) and the other one locks the screen in, I cant move and mouse movement wont rotate the view (locked camera/sort of a menu camera without menu). In locked mode I would be able to select the square under pieces and move them to a different one through a ray Im casting into the scene to my cursor position.

What I Have:

Camera class that is working as intended made according to this code from learnopengl.com https://learnopengl.com/code_viewer_gh.php?code=includes/learnopengl/camera.h

A mouse ray class made according to this tutorial: https://antongerdelan.net/opengl/raycasting.html

Mouse direction calculation put into code (the last function returns the direction correctly as Ive tested it with a locked camera)(This is update with every render):

    glm::vec3 NormalizedDeviceCoordinates() {
        float x = (2.f * this->mouseX) / this->frameBufferWidth - 1.0f;
        float y = 1.0f - (2.0f * this->mouseY) / this->frameBufferHeight;
        float z = 1.0f;
        return glm::vec3(x, y, z);
    }

    glm::vec4 HomogeneousClipCoordinates(glm::vec3 NormalizedDeviceCoords) {
        return glm::vec4(NormalizedDeviceCoords.x, NormalizedDeviceCoords.y, -1.f, 1.f);
    }

    glm::vec4 EyeCoordinates(glm::vec4 HomogenousClipCoords) {
        glm::vec4 ray_eye = glm::inverse(projectionMatrix) * HomogenousClipCoords;
        return glm::vec4(ray_eye.x, ray_eye.y, -1.f, 0.f);
    }

    glm::vec3 WorldCoordinates(glm::vec4 EyeCoords) {
        glm::vec3 ray_wor = (glm::inverse(viewMatrix) * EyeCoords);
        ray_wor = glm::normalize(ray_wor);
        return ray_wor;
    }
    
    glm::vec3 calculateMouseRay() {
        return WorldCoordinates(EyeCoordinates(HomogeneousClipCoordinates(NormalizedDeviceCoordinates())));
    }

I also have a keyboard input function with which I can switch between the 2 modes upon pressing M key. Differences between unlocked and locked mode:

• glfwSetInputMode(this->window, GLFW_CURSOR, GLFW_CURSOR_DISABLED) / glfwSetInputMode(this->window, GLFW_CURSOR, GLFW_CURSOR_NORMAL)
• allows for camera movement and rotation / does not allow movement or camera rotation

The problem:

I genuinly dont know how to describe it so I made a video and uploaded it to youtube.

Here is the link: https://youtu.be/4s-M6vHxvCc

Now, time for explaining:

• The black and white cube you see is my attempt at tracking the direction ray, (simply put I draw the cube and send transformation matrix to the shader which transforms it to camera location + ray direction vector), for now the cube is shown in both modes (unlocked and locked).
• In the first half of the video Im in the unlocked mode. You can see me trying to rotate to left and right in an attempt to show that the cube is somewhat stuck in this angle and wont move beyond it.
• In the second half of the video (after the cube "snaps" to position) I switch to locked mode. You can also see my cursor as well as the cube. They arent alined but the cube is in fact replicating cursor movement. (It is also worthy to point out that there is offset between cursor and the position, likely due to previous mouse rotation from unlocked mode, idk how to account for that)

Possible solutions/reasons/ideas:

Im not completely certain on this, but I think that there are more than one problems overlapping.

First thing first I think the cube is somehow locked to horizontal plane (unintentionally). If I move with movement keys the cube moves along with me but whenever I move the mouse it is always moving along that one plane.

Secondly, it logically should not be horizontal plane but plane to which the first screen was rendered (English is not my native language).

Thirdly If the second assumption is correct I need to somehow move this plane according to mouse rotation (it might be better to say camera direction) which I dont know how to do (or at which point I should add this to the equation at all in fact).

Afterword:

You may have noticed the problem with the first half of the video (unlocked mode). Put to video game terms (this is an assumption) I have both the fps camera (which usually has "cursor" in the middle of the screen) and some sort of menu camera (which traces the real cursor position) working at the same time, which is bad because what I would want is the cube being drawn in the center of the screen (camera direction) AND only when I switch to (locked mode) would the cube start moving based of off cursor position. But for example purposes you should see that there is a different problem mentioned above.

I will be grateful to anyone who can answer the question or point me to the right direction, if you need more information ask away.

Answer 1

等待大神答复