{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# ARTag Localization\n", "\n", "In this tutorial we will continue to explore how the various sensors on board the Intel Aero Drone can provide information on where the drone is in the world (i.e. localizations), which is a critical first step to autonomous navigation. Here we will explore the use of visual fiducial known as an [ARTag](https://en.wikipedia.org/wiki/ARTag). See below for an example.\n", "\n", "![Source: http://wiki.ros.org/ar_track_alvar](http://wiki.ros.org/ar_track_alvar?action=AttachFile&do=get&target=artags.png)\n", "\n", "In the [optical flow](enable_optical_flow.html) tutorial, we saw how the combination of the TeraRanger distance sensor, downward facing camera, and \"black box\" service/software `aero-optical-flow` enabled the new flight mode of `POSITION CONTROL` which was significantly easier to pilot than `MANUAL`. \n", "\n", "This tutorial will similarly use an onboard camera and pre-compiled libraries/software to provide a form of positioning information. In contrast to the optical flow tutorial, we will be using the front-facing [RealSense](https://en.wikipedia.org/wiki/Intel_RealSense) camera suite and the `ar_track_alvar` ROS package (see [here](http://wiki.ros.org/ar_track_alvar) for more details)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Launching RealSense & `ar_track_alvar`\n", "To launch both the forward-facing camera and `ar_track_alvar`, you will need to use ROS's built in `roslaunch` tool. We have already set up a launch file that does both of these for you. \n", "\n", "### roslaunch\n", "\n", "Syntax: `roslaunch .launch []`\n", "In this case, the package will be `aero_control` and the launch file will be `ar_visualization.launch`.\n", "Go ahead and run `roslaunch aero_control ar_visualization.launch`. This should start both the camera and `ar_track_alvar`. \n", "\n", "### `ar_track_alvar` topics\n", "\n", "`ar_track_alvar` publishes the following topics:\n", "\n", "* /visualization_marker\n", "* /ar_pose_marker\n", "\n", "See if you can determine what type of message is published on each of these channels using `rostopic info /` for both. After doing that, you might try `rostopic echo /` and holding an AR tag in front of the drone. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Visualizing with rviz\n", "On the drone in desktop mode, type `rviz` in an open Terminal window. The ROS visualization software will then open. You will need to select certain options to make `rviz` work properly.\n", "1. Select Fixed Frame in the top left corner of rviz and change it from `map` to `camera_rgb_optical_frame`\n", "![frame-select.png](https://github.com/BWSI-UAV/website/blob/master/docs/images/frame-select.png?raw=true)\n", "2. Click Add in the lower left corner of rviz.\n", "![add-rviz.png](https://github.com/BWSI-UAV/website/blob/master/docs/images/add-rviz.png?raw=true)\n", "3. Add a Camera Display from `/camera/rgb/image_color`. \n", "![add-camera-rviz.png](https://github.com/BWSI-UAV/website/blob/master/docs/images/add-camera-rviz.png?raw=true)\n", "4. Add a TF display.\n", "![add-tf-rviz.png](https://github.com/BWSI-UAV/website/blob/master/docs/images/add-tf-rviz.png?raw=true)\n", "5. Deselect \"All Enabled\" under \"Frames\" in the TF display options. (You can also resize/move your Camera Display to be more visible)\n", "![disable-frame-rviz.png](https://github.com/BWSI-UAV/website/blob/master/docs/images/disable-frame-rviz.png?raw=true)\n", "6. Move an AR tag into frame and watch the label/axes change as you move it around/twist it. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Processing Marker\n", "\n", "In preparation for this week's challenge, you will need to write your own ROS node that subscribes to the appropriate ARTag topics, parses important information from messages, and displays instructions to the [Ground Commander](team_roles.html#Ground-Commander) to aid in flight maneuvers.\n", "\n", "First let's make sure you have the required libraries installed on your drone/computer:\n", "\n", "```\n", "sudo apt install ros-kinetic-ar-track-alvar\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First, pull from upstream in `aero_control`. Your `aero_control` directory should now have a files called `ar_localization/src/ar_checker.py` that has blanks for you to fill in. Your goal with this file is to read AR tag detections from `ar_track_alvar` and check to see that the drone is within the proper range from the tag to be counted as a detection (by default, 0.9 - 1.1m but we may change this if it proves too easy). Your code should then log the detection or, if not in the proper position, directions on getting to the proper position using the information from `ar_track_alvar`. It should then log the tag as \"seen\" and display an \"Already Seen\" message if it appears again." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You will be able to test your `ar_checker.py` using `ar_checker.launch`, also in your `aero_control` directory. When roslaunching, you can use `roslaunch aero_control ar_checker.launch 2>/dev/null` to get rid of the error stream that MAVROS puts out." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Testing with bag files\n", "We can test the `ar_checker.py` using a ROS bag file (indicated by a `.bag` extension). \n", "Syntax for rosbag: `rosbag play .bag`. \n", "\n", "Link for `ar_checker.bag`: https://goo.gl/uDMUxz \n", "\n", "Testing procedure for rosbag:\n", "\n", "1. Run `roscore &` in Terminal on your VM.\n", "2. Open another Terminal window and run `rosbag play ar_checker.bag` in the proper directory (likely `~/Downloads`).\n", "3. Run your `ar_checker` node in a third Terminal window by typing `python ar_checker.py` while in the proper `aero_control` directory.\n", "4. You can visualize the images in the bag file using `rqt_image_view`. Note: the detections will not start to appear until ~10 seconds into the 30 second bag file. \n", "\n", "\n", "\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 2", "language": "python", "name": "python2" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.15+" } }, "nbformat": 4, "nbformat_minor": 2 }