Saturday, April 21, 2012

Detecting the type of mouse click

There are many ways for a user to interact with application using a mouse, there are normal click (single left click) that often selects the object, single right click that often opens up a context menu, and double-left click that often opens up the object. The ability to detect the type of the mouse click will come in handy when you are developing a truly interactive GUI. Fortunately, such information can be easily obtained by extracting the information stored in the SelectionType property of the GUI, which is a figure object.

1. Create a blank GUI and save the files.

2. Create a function to be attached to the WindowButtonDownFcn property of the GUI.


function winButtonDownFcn(hObject, eventData) 
m_type = get(hObject, 'selectionType');
if strcmp(m_type, 'normal')
    fprintf('left mouse click\n');
elseif strcmp(m_type, 'alt')
    fprintf('right mouse click\n'); 
elseif strcmp(m_type, 'open')
    fprintf('double-click\n');
end

3. In the opening function of the GUI, attach the function to the WindowButtonDownFcn property of the GUI.

set(hObject, 'windowButtonDownFcn', @winButtonDownFcn);

4. Running the GUI and try out different type of mouse clicks, the corresponding detection should be printed in the command window.

Note that there are other mouse clicks and/or key pressed may result in the same values in the selectionType property, for example, Control+left click will show up as a right click. If you would like to implement the more complicated mouse clicks (with key pressed), be sure to check the table in the documentation.

Example can be download here.

Monday, April 16, 2012

Making multiple objects drag-and-droppable

To continue on with the drag and drop topic, we now look at a way to make multiple objects in the gui drag-and-droppable. The idea is simple, rather than focusing on one object, we will be creating a cell array that stores all the handles to the objects that we would like to move, so that depending on where the mouse is, different objects will be the target of the drag-and-drop action. If we don't consider the case where multiple objects overlap, making multiple objects movable is a fairly straightforward procedure. However, to account for the fact that the different objects may overlap, the decision as to which object should be considered first, as well as the implementation of such decision make this problem interesting.

Note that I am making things a little complicated here because my starting point is windowMouseMotionFcn instead of the buttonDownFcn of the specific object. The former detects mouse motion while the latter detects mouse click on the specific object. I could easily start at buttonDownFcn and have the gui behave almost the same way, except that you won't be able to tell if an object is movable unless you click on it, in other words, there won't be any indicator (a hand-shape figure pointer) to indicate that the object is movable. Starting from the windowMouseMotionFcn is, in a way, a walk-around to the missing mouseOver fucntion, which allows you to implement the 'movable indicator'. That said, if you don't care about this functionality, check out selectmoveresize function, which is a MATLAB function that allows you to select/move/resize axes and uicontrol graphic objects.

Key procedures:

1. Create three static text fields and name their tags as lbl_target_1, lbl_target_2, and lbl_target_3. Change their unit to pixel. You can also change their background color to green and fontsize to 16.

2. In the opening function of the gui, create a cell array that consists of all the handles to the static text fields. The order of the handles in the cell array should reflect the order of corresponding uicontrols in the gui, specifically, if you wish to have a particular uicontrol appearing on top of another when they overlap, make sure its handle comes before the handle of the other uicontrol in the array.

movables = {handles.lbl_target_1, handles.lbl_target_2, handles.lbl_target_3};



3. Using uistack function to re-arrange the order of the uicontrols. First move the uicontrol to the top level, then move them down by an order that is equal to their position in the array minus 1. For example, for the third uicontrol, move it to the top first, then move it down by 2.

for i = 1:length(movables)
    cur_target = movables{i};
    uistack(cur_target, 'top'); 
    uistack(cur_target, 'down', i-1);
end


4. Attach drag_and_drop function to the windowMouseMotionFcn, with the argument movables as the cell array that contains the handles to desired uicontrols.

set(hObject, 'WindowButtonMotionFcn', {@drag_and_drop, movables});
5. In the drag_and_drop function, which is called whenever the mouse is moved on top of the gui, use a for-loop to go through all the targets and find out whether the mouse is on top of any of the uicontrol in the supplied handles array. Any time the mouse is found on top of a uicontorl, record the index of the uicontrol in the array and break the loop. Note that since the uicontrols have been sorted based on their "stack order", if there are multiple uicontrols overlap, the one at the top will always be detected.

target_id = [];
% find out which one the mouse is on top of right now 
for i = 1:length(target_handles)
    % get position information of the uicontrol
    lbl_target = target_handles{i};
    bounds = get(lbl_target,'position');
    lx = bounds(1); ly = bounds(2);
    lw = bounds(3); lh = bounds(4);
    if (x >= lx && x <= (lx + lw) && y >= ly && y <= (ly + lh))
        target_id = i;
        break;
    end
end

6. Once the index of the uicontrol is known, use another for-loop to attach grab function to the corresponding buttonDownFcn of the target uicontrol. A for-loop is needed because some uicontrols may require 'resets' if they are shadowed by another higher order uicontrol. .

for i = 1:length(target_handles)
    lbl_target = target_handles{i};
    if target_id == i 
        % set enable to off so that the whole static text field is hotspot
        set(lbl_target, 'enable', 'off');
        set(lbl_target, 'string', 'IN');
        set(lbl_target, 'backgroundcolor', 'red');
        set(lbl_target, 'ButtonDownFcn', @grab);
        setfigptr('hand', handles.fig_mouse);
    else
        % re-enable the uicontrol
        set(lbl_target, 'enable', 'on');
        set(lbl_target,'string', 'OUT');
        set(lbl_target, 'backgroundcolor', 'green');
    end
end
if isempty(target_id)
    setfigptr('arrow', handles.fig_mouse);
end

Video demo:



Files for download

Note: 
1. in the functions attached to the different callbacks of mouse actions in the gui, make sure you are not referring to other uicontrols in the gui by their tags, doing so will limit the use of the functions to the specific gui only. My drag_and_drop function will be a good example of this bad practices, since I  refer to the labels that reflect mouse action/position using their tags. 
2. Static text field isn't really a good choice as the uicontrol to be move around. In future posts in the series, I will switch to axes object instead. 

Monday, April 9, 2012

Sharing data among MATLAB GUIs: Part 2

In Part 1 of the series, I have shown that how data can be shared between the guis by writing sub-functions in the gui to take over the responsibility. Even though the idea is simple, the actual implementation takes some skills and understanding of how MATLAB guis work.

Often time, you will see the use of global variables proposed as a quick and easy fix to the problem. If the design of the application is simple, I can see why it may be the preferred approach. However, the main issue I have with the use of global variables is its awkward use.

When you use global variables, you are forced to assign/read data to/from global variables instead of push/pull data. Why does it matter, you may wonder. Well, it is awkward. When you have two guis running at the same time, the user expects that the action performed in one gui should be sufficient to complete the data sharing, rather than interacting with two guis. For example, in the example I gave in Part 1, depending on which button the user clicks, the data will either be pulled or pushed to the target gui and displayed in the text box. If you were to use global variable, you will need to first save the string in a global variable and then load the string from the global variable in the target gui, which requires two actions from the user. Isn't that awkward for data sharing? That's just for two guis, if you have five guis, you will have to cycle through all five of them to have the most up-to-date version of the data in all five of the guis.

Not only more actions is required from the user, the order of the actions is also important. If the order is not followed properly, the results in the gui may not be what's expected. For example, if you read from the global variable in the target gui before assigning updated value to the global variable in the source gui, then the target gui has the outdated values.

Finally, if you wish to expand the existing application or combine it with other guis, there may be conflicts as to how the global variables are used. I can't imagine how much of a pain that is going to be (Really, I haven't done it that way, so I have no idea).

The above are just my opinions on why using global variable is not the appropriate mean to address the data sharing among MATLAB guis. That said, I am not an expert, so if anyone thinks I am talking non-sense here, please do let me know.



Sunday, April 8, 2012

Sharing data among MATLAB GUIs: Part 1

When there are multiple GUIs in your MATLAB application, the need to let the GUIs communicate with each other is inevitable. While sharing data between the different callbacks within the same GUI may already turn some people off, the sharing of data among different GUIs is not as difficult as one may think.

In short, you need to write functions in the different GUIs to deal with establishing communication links and data exchange, and when the time comes to pull/push data to/from a GUI, simply call the appropriate functions via the GUI's functions to have the data updated.

When two GUIs are sharing data, one of them must be providing the data while the other must be receiving the data. For simplicity'sake, I am going to call the one that sends the data as the source gui and the one that receives the data as the target gui.

In the source gui, you need to have a function that returns the desired data when it is called, say, get_data(). Similarly, in the target gui, you need to have a function that read in (or use) the data, say, set_data().

Depending on where you initiate the data exchange between the two guis, the sequence of actions is different. If the communication is initiated at the source gui, you are pushing data to the target gui; on the other hand, if it is initiated at the target gui, you are pulling data from the source gui. The former case will call the set_data function in the target gui to have the data pushed over, while the latter case will call the get_data function in the source gui to have the data pulled from the source gui.

Getting down to the actual implementation, there are basically two steps. In the first step, you need to make the guis aware of each other's existence and know what their responsibility is; in the second step, you need to specify the detail/protocol of the communication. You can think of the first step as the laying down the necessary hardware and the second step as implementing software that utilize the hardware in place.

Step 1: Setting up the communication links between the guis

To share data between guis, they have to be aware of each other. In the simplest cases where you have a source gui and a target gui, you will need to have the handle of the target gui in the source gui, and the handle of the source gui in the target gui. To accomplish that, you can have a set_target function in the source gui and a set_source function in the target gui. When the functions are called, they will take the handle of the other gui and store it as an item in the handles structure for later use. Below is a sample set_target function in the source gui.


function set_target(source_handle, target_handle)
% retrieve handles structure in the source gui
handles = guidata(source_handle);  
% assign target handle as an item in the handles structure 
handles.target = target_handle;
% save the handles s
guidata(source_handle, handles);


With the set_target and set_source functions properly written in the source gui and target gui, running the following code will have the communication link established.

% This will be your main script 
% start up the two guis 
source_handle = source_gui;
target_handle = target_gui; 
% establish the link between the two 
source_gui('set_target', source_handle, target_handle); 
target_gui('set_source', target_handle, source_handle);

Step 2: Setting up the communication protocol (well, in a way)

Once the links are established, you will need to implement rules as to what and how the data is shared. During the design, you need to make sure the guis make no assumption about what is going on in the other gui or what variables are being used. Knowing exactly what your intention is with each gui should be able to help you set up your rules properly.

As an example, I have two guis with text boxes and push buttons in them. The push button in the source gui is labelled 'Send', while the one in the target gui is labelled 'Retrieve'. The interaction is fairly straightforward, write whatever you can think of in the text boxes, once a push button is clicked, you will see the text in the source gui copied over to the text box in the target gui. That said, clicking the Send button will push the data, while clicking the Retrieve button will pull the data. Same results, different approach though. The set_data and get_data functions are posted below. Nothing fancy, they either set the string property or get the string from the text box.

function set_data(handles, s) %  in the target gui 
set(handles.txt_tgt,'string', s);


function s = get_data(handles) % in the source gui 
s = get(handles.txt_src,'string');

Note that to be able to call these functions from a different guis, you need to first `publish' the functions by adding the handle to the function as an item in the handles structure. To do that, in the opening function of the gui (before the handles structure is saved), add

handles.set_data = @set_data;

to the target gui, and add

handles.get_data = @get_data; 

to the source gui. Doing so will allow you to access these functions from a different gui.

As the final step, we just need to set up the callbacks of the push buttons, so that the data exchange will occur when the button is clicked. And it is in these functions, we will call the set_data and get_data function we prepared earlier.

% in the target gui 
% --- Executes on button press in pb_retrieve.
function pb_retrieve_Callback(hObject, eventdata, handles)
% get the handle to the source gui 
source_handle = handles.source; 
% get the handles structure of the target gui 
s_handles = guidata(source_handle); 
% update string to the one returned by the get_data function 
set(handles.txt_tgt, 'string', s_handles.get_data(s_handles));

% in the source gui 
% --- Executes on button press in pb_send.
function pb_send_Callback(hObject, eventdata, handles)
% get the handle to the target gui 
target_handle = handles.target; 
% get the handles structure of the target gui 
t_handles = guidata(target_handle); 
% call the set_data function 
t_handles.set_data(t_handles, get(handles.txt_src, 'string'));


Now, if you run the 'main' script to have the two guis running at the same time and have their link established, you will be able to observe the string passing from the source to the target. The example can be downloaded here. Run the test_script.m file.

While the example used is a very simple two-gui with one-way data flow, the same concept can be expanded and applied to cases where you want one-to-many or many-to-many guis with two-way communications. As long as you can have a clear design before diving into the coding, things should be relatively straightforward.

Saturday, April 7, 2012

The reason why uicontextmenu is not working with imshow()

When you ask why the uicontextmenu is not working in an axes plotted with an image, most people will say that it is the hittest property of the image object. If the hittest property of the image object is set to 'on', any time you click the mouse within the axes, the image object, which covers the entire axes, will take over the click action and deal with it. Since you probably don't have anything associate with the image object, it will appear doing nothing at all. But at the same time you are expecting the uicontextmenu from the axes to do something. I know, drives you crazy.

If you create the image with the image() or imagesc() function, setting the hittest property of the image object to 'off' should fix the problem. Effectively, it allows you to click through the image and get to the axes.

However, if you create the image with the imshow() function, you are faced with the same problem, the uicontextmenu on the axes doesn't do anything. You can spend hours searching for a logical explanation on the Internet, and yet nothing turns up.

Well, hopefully the few hours I wasted can save you some time.

The reason is actually quite simple. One obvious differences between the images created from image() and imshow() is the ticks on the axes, while image() shows it, imshow() hides it. Now, think of the ways you can use to get rid of the ticks, you can set xtick and ytick to '[]' using set function, or you can just set the entire axes to invisible. Guess what MATHWORKS did? THEY SET THE ENTIRE AXES TO INVISIBLE, so even if you click through the image object, you won't be able to click on the axes BECAUSE IT IS INVISIBLE.

So here is the fix.
Besides setting the hittest property of the image object to 'off', you need to set 'visible' property of the axes to 'on'. Once you do that, the xticks and yticks will appear, but you can easily make them disappear by setting them to [].

Friday, April 6, 2012

Interactive drawing in MATLAB: Part 2

In the previous post, we look at how we can draw different shapes on an axes using context menu and the MATLAB build-in function such as imline, imrect, etc.

The nice thing about the build-in functions is that they are fully implemented, meaning that they can be resized, dragged, and they even have context menu on their own which allows one to change the color and some other properties.

However, one thing I hate about the functions is the little rectangle boxes appear as the end points. I understand they are there to show you where you can click to move the points, but they should at least provide an option to turn the rectangle boxes visible or invisible. I fail to see any benefits of having the rectangle boxes on the final print out. I tried and failed to find any ways to remove the rectangle boxes, so in this post, we are going to code our own function for drawing the different shapes, without the annoying rectangles. (TAKE THAT!!! MATHWORKS).

The context menu created in the previous post can be re-used here. Instead of calling the imline or imrect functions when the corresponding menu items are selected, we are just going to create our own function to draw the desired shapes. Note that at this point, we will not be worry about all the fancy stuff, such as the figure pointer, line color, ability to resize, etc.

Design of the action sequence:
1. User selects "Line" from the context menu of the axes, which activates the line drawing mode.
2. Whenever the button is pressed in the line drawing mode, a line object is created on the axes.
3. As the user moves the mouse on the axes while holding the button down, the line object is continuously updated with the current location as the end point.
4. Whenever the users release the button, the line object is finalized and the line drawing mode ends.

The draw_line function shown at the bottom can fulfill the action sequence described above. Note that we are using WindowButtonDownFcn, WindowButtonMotionFcn and WindowButtonUpFcn to track the various mouse actions, and the corresponding GUI response is realized using nested/sub functions. You can try out this function easily using the code "plot(rand(1,10)); draw_line(fh,ah);". To implement functions that draw other shapes, the same structure can be followed, and changes only have to be made for the drawing of the shapes, which can be complicated at times depending on what you want to draw. 

Video demo



The sample GUI can be downloaded here.


function draw_line(fh, ah)
% DRAW_LINE(FH, AH) draws a line on an existing axes interactively. 

% Input: 
%   fh - handle of the figure that contains the axes 
%   ah - handle of the axes to be drawn on 
%
% Wei Shang 
% wei.shang@unb.ca 
% University of New Brunswick 


% get the boundaries on the axes 
x_bounds = get(ah, 'xlim');  y_bounds = get(ah, 'ylim');
% enter line drawing mode by attaching buttonDown function
set(fh, 'WindowButtonDownFcn', @buttonDown);


    function buttonDown(fh, dummy)
        % This function is called when the mouse button is pressed. 
        % It will create a line object based on the location of where the
        % button pressed action occured. 
        
        % get the location of the action 
        p = get(ah,'currentpoint');
        x1 = p(1,1); y1 = p(1,2);
        % create a line object that starts and ends at the same point. 
        lh = line([x1,x1], [y1,y1], 'linestyle', ':');
        % atttach buttonMove function 
        set(fh,'WindowButtonMotionFcn',@buttonMove);
        % attach buttonUp function
        set(fh,'WindowButtonUpFcn', @buttonUp);
        
        function buttonMove(fh, dummy)
            % This function is called when the mouse button moved while
            % being pressed. 
            
            % obtain end points of the line 
            X = get(lh,'XData'); Y = get(lh,'YData');
            
            % obtain new points as the new mouse pointer location 
            p = get(ah,'currentpoint');
            x2 = p(1,1); y2 = p(1,2);
            
            % check and make sure they are within the bounds, if not force
            % the end point to be within the bound 
            if x2 < x_bounds(1)
                x2 = x_bounds(1);
            elseif x2 > x_bounds(2);
                x2 = x_bounds(2);
            end
            
            if y2 < y_bounds(1)
                y2 = y_bounds(1);
            elseif y2 > y_bounds(2)
                y2 = y_bounds(2);
            end
            
            % update the line object with the 
            X(2) = x2;   Y(2) = y2;
            set(lh, 'XData', X, 'YData', Y);
            
        end % ends buttonMove function 
        
        function buttonUp(fh, dummy)
            % This function finalize the line object and ends line drawing
            % mode. 
            
            % set line object's line style to solid line 
            set(lh,'linestyle','-');
            
            % set various mouse actions to null
            set(fh, 'WindowButtonMotionFcn', '');
            set(fh, 'WindowButtonUpFcn', '');
            set(fh, 'WindowButtonDownFcn', '');
            
        end % ends buttonUp function 
        
    end % ends buttonDown function 


end % ends draw_line function


Interactive drawing in MATLAB: Part 1

Being able to draw in the plots/images in MATLAB GUI can be very useful at times. 

MATLAB has a collection of Utilities for Interactive Tools which allows you to place points, lines, rectangles, ellipses, etc on to an axes, so be sure to check them out first to see if they meet your need. 


There are many ways to incorporate/initiate/activate the various drawing tools in MATLAB GUI. You can simply call the functions in the callback of the various uicontrols such as push buttons, drop-down menu, etc. 


In the example that I am about to show, I attach a context menu to the axes, which allows the user to select the desired shape and begin drawing.

1. Create a new MATLAB GUI using the GUI with Axes and Menu template.

2. In the GUIDE window, bring up the Menu editor by Clicking Tools -> Menu Editor.

3. By default, the menu window is in Menu Bar mode, select Context Menus tab at the button of the menu editor.

4. Create the menu items as follows
    a. create a top level menu and name* it cm_axes, this menu item is to be attached to the axes
    b. under the cm_axes menu, create a new menu item and name it cm_draw, this menu item acts as a container of all the possible shapes that you can draw.
    c. under the cm_draw menu, create a new menu item and name it cm_draw_line, once selected, this menu item should allow the user to draw a line
    d. again, under the cm_draw menu, create a new menu item and name it cm_draw_rect, once selected, this menu item should allow the user to draw a rectangle.
    e. repeat d to implement another other shapes that you would like to use.

* By naming a item, I meant changing the item's tag, not labels. The labels of the items can be anything as long as it makes sense.

5. Once the context menu is created, the next step is to attach the context menu to the axes. To do that, simply open up the property editor of the axes, find UIContextMenu, and select cm_axes from the drop down menu.

6. After saving the file, the various callbacks for the menu item should appear at the bottom of your m-file editor. For the menu items cm_draw_line and cm_draw_rectangle, simply insert the two functions imline and imrect, respectively.

7. Save all file, and run the GUI, right click anywhere on the axes to bring up the context menu.

Video walkthrough



Source File

Download the files here.