Integration: Comprehensive Integration Testing

[nguy8tri]

Description

We need to more thoroughly test the Distance pipeline with many more tests, which must also more complex than is currently available. The issue with doing so however is that we need to not rely on existing infrastructure to generate and validate more complex scenarios. The scenarios that we have not tested so far are:

1. Complex Positions (Arbitrary Axis Positions)
2. Complex Orientations (Arbitrary Orientations)
3. Complex Relative Orientations (Arbitrary Transformations between local and reference orientations)
4. Combined Scenarios (of the above 3)

This task will involve the modification of test/integration/integration-test.cpp, and at your discretion, you may also choose to implement a new feature. That's it, so don't get scared by the big description here, its just to help you be familiar with the problem. I will attempt to address each of these:

Before Getting Started...

We use the image generator for generating artificial, ideal images. This allows us to test the algorithm to ensure it gives a precise answer when in an ideal environment. Please go into tools/generator for more information.

Warning

Also, please familiarize yourself with the different flags in the FOUND executable (we will just say FOUND) via ./found --help. You must also understand how FOUND works in general.

1. Complex Positions

So far, we have only tested positions on the X axis during our integration tests. The task here is simple:

1. Generate images from more complex positions
2. Create integration tests to see if FOUND can correctly produce the position.

Note

You may use the flag --reference-as-orientation here, which you can use in conjunction with --reference-orientation to directly tell FOUND the orientation of the camera.

Warning

These tests must be marked as being seperate from (1), (2) and (3). You may use a simple orientation here.

2. Complex Orientations

So far, we have tested very basic orientations (only right ascension). The task here is also simple:

1. Generate images with more general orientations
2. Create integration tests to see if FOUND can correctly produce the position while handling a more complex orientation.

Note

Again, you may use --reference-as-orientation here.

Warning

These tests must be clearly marked as being seperate from the above. You may use a simple position here.

3. Complex Relative Orientations

Because of #39 , this is no longer complex (refer to #40 (comment) in case the current approach does not work). Thanks to tools.attitude, this has become much simpler:

1. Use tools.generator to generate an image with a complex orientation.
2. Use tools.attitude (remember to switch your conda environment) to generate attitudes for calibration and a test reference attitude for distance, using the --use-local and --local-attitude flags.
3. Format your test cases to be like IntegrationTest::TestCalibrationDistanceCombinedPipeline IntegrationTest::TestCalibrationDistanceCombinedPipelineOtherOutput, located in the integration-test.cpp file.

4. Combined Scenarios

As the good programmer you are, you will also make test cases that combine these different things :).

[nguy8tri]

Appendix:

3. Complex Relative Orientations

Background

The design of FOUND is such that it can work with a reference orientation that does not represent its own orientation, just as long as the orientation of FOUND is fixed relative to that reference orientation and we know how to go from one to the other.

FOUND currently does this by accepting the orientation of both a reference and itself, and stores the rotation needed to go from the reference to its own orientation (what we call the local orientation). This "relative rotation" is calculated during the Calibration pipeline, and is stored in a data file.

During the Distance Pipeline stage, you can pass both the reference orientation and the data file, and the distance pipeline calculates the local orientation. It does this all using Quaternions, a 4 dimensional complex number space that can easily execute rotations in 3D real space.

How to test for this

However, we should not use Quaternions again to verify that this particular way of calculating the orientation is correct (good for you, no need to understand it). We should use an alternate way to cross-check it. There are probably many ways, but the easiest way to do this is to manually calculate a direction cosine matrix:

1. You generate a randomized 3D axis relative to the normal 3D basis (<1, 0, 0>, <0, 1, 0>, <0, 0, 1>). These are the initial local and reference axes. You can literally just randomly select your new x axis. Then select y to be perpendicular to x (you can use the formula for a cross product here). Then, you generate z by crossing x and y (in that order). Make sure everything has been normalized
2. You then generate the Direction Cosine Matrix (DCM), which is a 3x3 matrix that holds the cosines between your axes and the normal 3D basis. If you've done step 1 right, the rotation matrix is just a column matrix of the 3D axes you just generated. If you're not using a reference axes that is the normal 3D basis, you simply multiply the transpose of the column basis of the reference axis by the column basis of the local axes.
3. By the same algorithm as 1, you generate new random axes. This is the new reference axis. Multiply a column matrix basis of these new axes by the DCM to obtain a new local axes. If you want, confirm that the new local axes have the same direction cosines by the alternate method above.
4. Taking the column bases for the initial and new axes (4 in total), use the equation below (in the image, at step 2) to obtain the RA, DE and ROLL corresponding to all axes. These are the Euler Angles that you will use next.
5. You will then use the new euler angles for the local axes to generate a new image of Earth (you may have to return to step 3 to get an orientation that will actually show Earth's edge).
6. You will use the initial Euler Angles and run them through the Calibration Pipeline to generate calibration data.
7. You will then run the distance pipeline passing in the calibration data with the new reference euler angles and the image. Hopefully, you will get whatever the position you used to generate the image with.

Here is a little descriptor about how to direction cosine matrix (DCM) works. Under (2), that conversion equation can apply to any of the 3 matrices to describe the rotation required to get to that coordinate system. The DCM to Euler Angle conversion taken from here (be mindful of the gimbal lock condition, which has the resolution shown in (2)). r is for reference and l is for local.

Finally, I will summarize the steps above:

You will find an example of how to make this test in IntegrationTest::TestCalibrationDistanceCombinedPipeline and IntegrationTest::TestCalibrationDistanceCombinedPipelineOtherOutput.

Generating Complex Orientation Data

You may find steps 1 through 5 something you'll need to repeat often. In fact, it is very possible we will have to use that particular generation algorithm again. You may find it useful to consolidate this logic into a python script, which could end up as a tool in the tools section. You may also choose to modify the generator tool to integrate with this algorithm.