c# - Using Unity3D's IPointerDownHandler approach, but with "the whole screen"

Question

In Unity say you need to detect finger touch (finger drawing) on something in the scene.

The only way to do this in modern Unity, is very simple:

---

Step 1. Put a collider on that object. ("The ground" or whatever it may be.) ¹

Step 2. On your camera, Inspector panel, click to add a Physics Raycaster (2D or 3D as relevant).

Step 3. Simply use code as in Example A below.

(Tip - don't forget to ensure there's an EventSystem ... sometimes Unity adds one automatically, sometimes not!)

---

Fantastic, couldn't be easier. Unity finally handles un/propagation correctly through the UI layer. Works uniformly and flawlessly on desktop, devices, Editor, etc etc. Hooray Unity.

All good. But what if you want to draw just "on the screen"?

So you are wanting, quite simply, swipes/touches/drawing from the user "on the screen". (Example, simply for operating an orbit camera, say.) So just as in any ordinary 3D game where the camera runs around and moves.

You don't want the position of the finger on some object in world space, you simply want abstract "finger motions" (i.e. position on the glass).

What collider do you then use? Can you do it with no collider? It seems fatuous to add a collider just for that reason.

What we do is this:

I just make a flat collider of some sort, and actually attach it under the camera. So it simply sits in the camera frustum and completely covers the screen.

(For the code, there is then no need to use ScreenToWorldPoint, so just use code as in Example B - extremely simple, works perfectly.)

My question, it seems a bit odd to have to use the "under-camera colldier" I describe, just to get touches on the glass.

What's the deal here?

(Note - please don't answer involving Unity's ancient "Touches" system, which is unusable today for real projects, you can't ignore .UI using the legacy approach.)

---

Code sample A - drawing on a scene object. Use ScreenToWorldPoint.

 using UnityEngine;
 using System.Collections;
 using UnityEngine.EventSystems;

 public class FingerMove:MonoBehaviour, IPointerDownHandler, IDragHandler, IPointerUpHandler
 {
     public void OnPointerDown (PointerEventData data)
     {
         Debug.Log("FINGER DOWN");
         prevPointWorldSpace =
                 theCam.ScreenToWorldPoint( data.position );
     }

     public void OnDrag (PointerEventData data)
     {
         thisPointWorldSpace =
                theCam.ScreenToWorldPoint( data.position );
         realWorldTravel =
                thisPointWorldSpace - prevPointWorldSpace;
         _processRealWorldtravel();
         prevPointWorldSpace = thisPointWorldSpace;
     }

     public void OnPointerUp (PointerEventData data)
     {
         Debug.Log("clear finger...");
     }

Code sample B ... you only care about what the user does on the glass screen of the device. Even easier here:

 using UnityEngine;
 using System.Collections;
 using UnityEngine.EventSystems;

 public class FingerMove:MonoBehaviour, IPointerDownHandler, IDragHandler, IPointerUpHandler
 {
     private Vector2 prevPoint;
     private Vector2 newPoint;
     private Vector2 screenTravel;

     public void OnPointerDown (PointerEventData data)
     {
         Debug.Log("FINGER DOWN");
         prevPoint = data.position;
     }

     public void OnDrag (PointerEventData data)
     {
         newPoint = data.position;
         screenTravel = newPoint - prevPoint;
         prevPoint = newPoint;
         _processSwipe();
     }

     public void OnPointerUp (PointerEventData data)
     {
         Debug.Log("FINEGR UP...");
     }

     private void _processSwipe()
     {
         // your code here
         Debug.Log("screenTravel left-right.. " + screenTravel.x.ToString("f2"));
     }
 }

---

¹ If you're just new to Unity: at that step very likely, make it a layer called say "Draw"; in physics settings make "Draw" interact with nothing; in step two with the Raycaster just set the layer to "Draw".

Answer 1

First of all, you need to understand that there are just 3 ways to detect click on an Object with the OnPointerDown function:

1.You need a UI component to in order to detect click with the OnPointerDown function. This applies to other similar UI events.

2.Another method to detect a click with the OnPointerDown function on a 2D/Sprite GameObject is to attach Physics2DRaycaster to the Camera and then OnPointerDown will be called when it is clicked. Note that a 2D Collider must be attached to it.

3.If this is a 3D Object with a Collider not 2D Collider, you must have PhysicsRaycaster attached to the camera in order for the OnPointerDown function to be called.

Doing this with the first method seems more reasonable instead of having a large collider or 2D collider covering the screen. All you do is to create a Canvas, Panel GameObject, and attach Image component that stretches across the whole screen to it.

Dude I just don't see using .UI as a serious solution: imagine we're doing a big game and you're leading a team that is doing all the UI (I mean buttons, menus and all). I'm leading a team doing the walking robots. I suddenly say to you "oh, by the way, I can't handle touch ("!"), could you drop in a UI.Panel, don't forget to keep it under everything you're doing, oh and put one on any/all canvasses or cameras you swap between - and pass that info back to me OK!" :) I mean it's just silly. One can't essentially say: "oh, Unity doesn't handle touch"

Not really hard like the way you described it. You can write a long code that will be able to create a Canvas, Panel and an Image. Change the image alpha to 0. All you have to do is attach that code to the Camera or an empty GameObject and it will perform all this for you automatically on play mode.

Make every GameObject that wants to receive event on the screen subscribe to it, then use ExecuteEvents.Execute to send the event to all the interfaces in the script attached to that GameObject.

For example, the sample code below will send OnPointerDown event to the GameObject called target.

ExecuteEvents.Execute<IPointerDownHandler>(target,
                              eventData,
                              ExecuteEvents.pointerDownHandler);

Problem you will run into:

The hidden Image component will block other UI or GameObject from receiving raycast. This is the biggest problem here.

Solution:

Since it will cause some blocking problems, it is better to make the Canvas of the Image to be on top of everything. This will make sure that it is now 100 blocking all other UI/GameObject. Canvas.sortingOrder = 12; should help us do this.

Each time we detect an event such as OnPointerDown from the Image, we will manually send resend the OnPointerDown event to all other UI/GameObjects beneath the Image.

First of all, we throw a raycast with GraphicRaycaster(UI), Physics2DRaycaster(2D collider), PhysicsRaycaster(3D Collider) and store the result in a List.

Now, we loop over the result in the List and resend the event we received by sending artificial event to the results with:

ExecuteEvents.Execute<IPointerDownHandler>(currentListLoop,
                              eventData,
                              ExecuteEvents.pointerDownHandler);

Other problems you will run into:

You won't be able to send emulate events on the Toggle component with GraphicRaycaster. This is a bug in Unity. It took me 2 days to realize this.

Also was not able to send fake slider move event to the Slider component. I can't tell if this is a bug or not.

Other than these problems mentioned above, I was able to implement this. It comes in 3 parts. Just create a folder and put all the scripts in them.

SCRIPTS:

1.WholeScreenPointer.cs - The main part of the script that creates Canvas, GameObject, and hidden Image. It does all the complicated stuff to make sure that the Image always covers the screen. It also sends event to all the subscribe GameObject.

public class WholeScreenPointer : MonoBehaviour
{
    //////////////////////////////// SINGLETON BEGIN  ////////////////////////////////
    private static WholeScreenPointer localInstance;

    public static WholeScreenPointer Instance { get { return localInstance; } }
    public EventUnBlocker eventRouter;

    private void Awake()
    {
        if (localInstance != null && localInstance != this)
        {
            Destroy(this.gameObject);
        }
        else
        {
            localInstance = this;
        }
    }
    //////////////////////////////// SINGLETON END  ////////////////////////////////


    //////////////////////////////// SETTINGS BEGIN  ////////////////////////////////
    public bool simulateUIEvent = true;
    public bool simulateColliderEvent = true;
    public bool simulateCollider2DEvent = true;

    public bool hideWholeScreenInTheEditor = false;
    //////////////////////////////// SETTINGS END  ////////////////////////////////


    private GameObject hiddenCanvas;

    private List<GameObject> registeredGameobjects = new List<GameObject>();

    //////////////////////////////// USEFUL FUNCTIONS BEGIN  ////////////////////////////////
    public void registerGameObject(GameObject objToRegister)
    {
        if (!isRegistered(objToRegister))
        {
            registeredGameobjects.Add(objToRegister);
        }
    }

    public void unRegisterGameObject(GameObject objToRegister)
    {
        if (isRegistered(objToRegister))
        {
            registeredGameobjects.Remove(objToRegister);
        }
    }

    public bool isRegistered(GameObject objToRegister)
    {
        return registeredGameobjects.Contains(objToRegister);
    }

    public void enablewholeScreenPointer(bool enable)
    {
        hiddenCanvas.SetActive(enable);
    }
    //////////////////////////////// USEFUL FUNCTIONS END  ////////////////////////////////

    // Use this for initialization
    private void Start()
    {
        makeAndConfigWholeScreenPinter(hideWholeScreenInTheEditor);
    }

    private void makeAndConfigWholeScreenPinter(bool hide = true)
    {
        //Create and Add Canvas Component
        createCanvas(hide);

        //Add Rect Transform Component
        //addRectTransform();

        //Add Canvas Scaler Component
        addCanvasScaler();

        //Add Graphics Raycaster Component
        addGraphicsRaycaster();

        //Create Hidden Panel GameObject
        GameObject panel = createHiddenPanel(hide);

        //Make the Image to be positioned in the middle of the screen then fix its anchor
        stretchImageAndConfigAnchor(panel);

        //Add EventForwarder script
        addEventForwarder(panel);

        //Add EventUnBlocker
        addEventRouter(panel);

        //Add EventSystem and Input Module
        addEventSystemAndInputModule();
    }

    //Creates Hidden GameObject and attaches Canvas component to it
    private void createCanvas(bool hide)
    {
        //Create Canvas GameObject
        hiddenCanvas = new GameObject("___HiddenCanvas");
        if (hide)
        {
            hiddenCanvas.hideFlags = HideFlags.HideAndDontSave;
        }

        //Create and Add Canvas Component
        Canvas cnvs = hiddenCanvas.AddComponent<Canvas>();
        cnvs.renderMode = RenderMode.ScreenSpaceOverlay;
        cnvs.pixelPerfect = false;

        //Set Cavas sorting order to be above other Canvas sorting order
        cnvs.sortingOrder = 12;

        cnvs.targetDisplay = 0;

        //Make it child of the GameObject this script is attached to
        hiddenCanvas.transform.SetParent(gameObject.transform);
    }

    private void addRectTransform()
    {
        RectTransform rctrfm = hiddenCanvas.AddComponent<RectTransform>();
    }

    //Adds CanvasScaler component to the Canvas GameObject 
    private void addCanvasScaler()
    {
        CanvasScaler cvsl = hiddenCanvas.AddComponent<CanvasScaler>();
        cvsl.uiScaleMode = CanvasScaler.ScaleMode.ScaleWithScreenSize;
        cvsl.referenceResolution = new Vector2(800f, 600f);
        cvsl.matchWidthOrHeight = 0.5f;
        cvsl.screenMatchMode = CanvasScaler.ScreenMatchMode.MatchWidthOrHeight;
        cvsl.referencePixelsPerUnit = 100f;
    }

    //Adds GraphicRaycaster component to the Canvas GameObject 
    private void addGraphicsRaycaster()
    {
        GraphicRaycaster grcter = hiddenCanvas.AddComponent<GraphicRaycaster>();
        grcter.ignoreReversedGraphics = true;
        grcter.blockingObjects = GraphicRaycaster.BlockingObjects.None;
    }

    //Creates Hidden Panel and attaches Image component to it
    private GameObject createHiddenPanel(bool hide)
    {
        //Create Hidden Panel GameObject
        GameObject hiddenPanel = new GameObject("___HiddenPanel");
        if (hide)
        {
            hiddenPanel.hideFlags = HideFlags.HideAndDontSave;
        }

        //Add Image Component to the hidden panel
        Image pnlImg = hiddenPanel.AddComponent<Image>();
        pnlImg.sprite = null;
        pnlImg.color = new Color(1, 1, 1, 0); //Invisible
        pnlImg.material = null;
        pnlImg.raycastTarget = true;

        //Make it child of HiddenCanvas GameObject
        hiddenPanel.transform.SetParent(hiddenCanvas.transform);
        return hiddenPanel;
    }

    //Set Canvas width and height,to matach screen width and height then set anchor points to the corner of canvas.
    private void stretchImageAndConfigAnchor(GameObject panel)
    {
        Image pnlImg = panel.GetComponent<Image>();

        //Reset postion to middle of the screen
        pnlImg.rectTransform.anchoredPosition3D = new Vector3(0, 0, 0);

        //Stretch the Image so that the whole screen is totally covered
        pnlImg.rectTransform.anchorMin = new Vector2(0, 0);
        pnlImg.rectTransform.anchorMax = new Vector2(1, 1);
        pnlImg.rectTransform.pivot = new Vector2(0.5f, 0.5f);
    }

    //Adds EventForwarder script to the Hidden Panel GameObject 
    private void addEventForwarder(GameObject panel)
    {
        EventForwarder evnfwdr = panel.AddComponent<EventForwarder>();
    }

    //Adds EventUnBlocker script to the Hidden Panel GameObject 
    private void addEventRouter(GameObject panel)
    {
        EventUnBlocker evtrtr = panel.AddComponent<EventUnBlocker>();
        eventRouter = evtrtr;
    }

    //Add EventSystem
    private void addEventSystemAndInputModule()
    {
        //Check if EventSystem exist. If it does not create and add it
        EventSystem eventSys = FindObjectOfType<EventSystem>();
        if (eventSys == null)
        {
            GameObject evObj = new GameObject("EventSystem");
            EventSystem evs =