Class: Heuristic
From MASH Wiki
The Heuristic class is declared in mash/heuristic.h.
To implement a new heuristic, you'll declare a new class that inherit from Heuristic. See the heuristics development page for detailed usage examples and information about the SDK.
Contents |
What is a heuristic?
A heuristic in the MASH project is an image feature extractor. Its job is to compute some numerical quantities, called features, given an image, the coordinates of a point in this image and the extent of a region of interest around that point:
The features
A feature is a floating-point number. Its meaning and value is heuristic-dependent: the MASH system makes no assumption about it. However, the overall purpose of features is to make the life of machine learning easier. This is generally achieved by including invariance in the heuristic, such that when the image change in a certain way (e.g. illumination, translation, rotation, etc.) that should not matter for the system, the computed features do not change.
Dimension of a heuristic
All the features computed by a heuristic are grouped in a features vector. The size of this features vector is called the dimension of the heuristic. This dimension can only depend on the size of the region of interest (in other words: as long as the size of the region of interest is the same, the dimension of the heuristic is the same for any point in any image).
Region of interest
A region of interest is a square area, centered on a point. It is defined by its extent and the coordinates of the point. Its dimensions (width and height) are given by: extent * 2 + 1 (in pixels).
Life-cycle of a heuristic
The methods that you must implement when writing a heuristic will always be called over the course of an experiment in the order shown below.
Sequences of images are relevant only for goal-planning. In that case there is a notion of time, and heuristics may implement some form of memory. For image classification and object detection, this loop can be ignored.
The visiting order of the loops is totally undefined. The same image or location may be visited several times, while others may not be visited at all.
Finally, the method dim() may be called at any moment over the life of the heuristic.
- init()
- Loop over sequences of images
- prepareForSequence()
- Loop over some images in the sequence
- prepareForImage()
- Loop over some locations in the image
- prepareForCoordinates()
- Loop over some feature indexes
- computeFeature()
- End loop over feature indexes
- finishForCoordinates()
- End loop over locations
- finishForImage()
- End loop over images
- finishForSequence()
- End loop over sequences
- terminate()
Attributes of the class
The Heuristic class has the following attributes, that you must use to implement your own heuristic:
class Heuristic
{
....
unsigned int nb_views; // Total number of views
unsigned int roi_extent; // Extent of the region of interest currently processed
coordinates_t coordinates; // Coordinates of the point currently processed
Image* image; // The image currently processed
};
With coordinates_t defined as follows:
struct coordinates_t
{
unsigned int x; // X position in pixels
unsigned int y; // Y position in pixels
};
Required methods
The following methods must be implemented by all the heuristics:
class Heuristic
{
....
public:
//----------------------------------------------------------------------
/// @brief Returns the number of features this heuristic computes
///
/// When this method is called, the following attributes are initialized:
/// - nb_views
/// - roi_extent
//----------------------------------------------------------------------
virtual unsigned int dim() = 0;
//----------------------------------------------------------------------
/// @brief Computes the specified feature
///
/// When this method is called, the following attributes are initialized:
/// - nb_views
/// - roi_extent
/// - image
/// - coordinates
//----------------------------------------------------------------------
virtual scalar_t computeFeature(unsigned int feature_index) = 0;
....
};
Optional methods
The following methods can be implemented by the heuristics that needs them:
class Heuristic
{
....
public:
//----------------------------------------------------------------------
/// @brief Called once at the creation of the heuristic
///
/// Pre-computes all the data that will never change during the
/// life-time of the heuristic
///
/// When this method is called, the following attributes are initialized:
/// - nb_views
/// - roi_extent
///
/// @remark The implementation of this method is optional
//----------------------------------------------------------------------
virtual void init() {}
//----------------------------------------------------------------------
/// @brief Called once when the heuristic is destroyed
///
/// Frees the memory allocated by the init() method
///
/// @remark This method must be implemented if init() is used and
/// allocated some memory
//----------------------------------------------------------------------
virtual void terminate() {}
//----------------------------------------------------------------------
/// @brief Called once per sequence, before any computation
///
/// Allocates and initializes the data the heuristic will need to
/// compute features on a stream of images (like a video)
///
/// When this method is called, the following attributes are initialized:
/// - nb_views
/// - roi_extent
///
/// @remark The implementation of this method is optional
//----------------------------------------------------------------------
virtual void prepareForSequence() {}
//----------------------------------------------------------------------
/// @brief Called once per sequence, after any computation
///
/// Frees the memory allocated by the prepareForSequence() method
///
/// @remark This method must be implemented if prepareForSequence() is
/// used and allocated some memory
//----------------------------------------------------------------------
virtual void finishForSequence() {}
//----------------------------------------------------------------------
/// @brief Called once per image, before any computation
///
/// Pre-computes from a full image the data the heuristic will need to
/// compute features at any coordinates in the image
///
/// When this method is called, the following attributes are initialized:
/// - nb_views
/// - roi_extent
/// - image
///
/// @remark The implementation of this method is optional
//----------------------------------------------------------------------
virtual void prepareForImage() {}
//----------------------------------------------------------------------
/// @brief Called once per image, after any computation
///
/// Frees the memory allocated by the prepareForImage() method
///
/// @remark This method must be implemented if prepareForImage() is used
/// and allocated some memory
//----------------------------------------------------------------------
virtual void finishForImage() {}
//----------------------------------------------------------------------
/// @brief Called once per coordinates, before any computation
///
/// Pre-computes the data the heuristic will need to compute features
/// at the given coordinates
///
/// When this method is called, the following attributes are initialized:
/// - nb_views
/// - roi_extent
/// - image
/// - coordinates
///
/// @remark The implementation of this method is optional
//----------------------------------------------------------------------
virtual void prepareForCoordinates() {}
//----------------------------------------------------------------------
/// @brief Called once per coordinates, after any computation
///
/// Frees the memory allocated by the prepareForCoordinates() method
///
/// @remark This method must be implemented if prepareForCoordinates()
/// is used and allocated some memory
//----------------------------------------------------------------------
virtual void finishForCoordinates() {}
....
};