im2im: Automatically Converting In-Memory Image Representations

---

The im2im python package provides an automated approach for converting in-memory image representations across a variety of image processing libraries, including scikit-image, opencv-python, scipy, PIL, numpy, PyTorch, and Tensorflow. It handles the nuances inherent to each library's image representation, such as data formats (numpy arrays, PIL images, torch tensors, and so on), color channel (RGB or grayscale), channel order (channel first or last or none), device (CPU/GPU), and pixel intensity ranges.

im2im uses the A* graph search algorithm to compute the smallest cost conversion path between image metadata. Each edge in the graph is assigned a cost tuple: (information loss, constraint score). The total cost of a path is evaluated lexicographically to prioritize minimal information loss. This ensures that the selected conversion is both functionally correct and optimal in preserving data integrity. You can find the A* implementation and cost evaluation logic here.

Usage

im2im is developed to simplify image type conversions in Visual Programming Systems (VPS) for image processing. It removes the need for manual conversion steps, significantly improving accessibility—especially for non-expert users.

Installation (At Execution Environment)

Install the package via pip:

pip install im2im

Or install the package directly from GitHub:

pip install git+https://github.com/c3di/im2im.git

Usage for Auto Type Conversion

API Entry Point: im2im

import numpy as np
from im2im import Image, im2im

# Example input: an image that is a numpy.ndarray with shape (20, 20, 3) in uint8 format
to_be_converted = Image(np.random.randint(0, 256, (20, 20, 3), dtype=np.uint8), "numpy.rgb_uint8")
# Convert to the target image with metadata preset "torch.gpu".
converted: Image = im2im(to_be_converted, "torch.gpu")

Metadata presets are defined in src/im2im/find_metadata_builtin_preset.py.

For additional APIs such as im2im_code and add new preset, please refer to src/im2im/api.py.

Integration in Visual Programming Systems:

Take the Blockly visual programming framework as an example. To enable automatic type conversion:

• Call im2im() before operations to convert the input.
• Explicitly set the metadata for the output image.

Without im2im

Blockly.Python.forBlock['gaussian_blur'] = function (block) {
  requiredImports.add("from skimage.filters import gaussian");
  // inputs
  var image = Blockly.Python.valueToCode(block, "IMAGE", Blockly.Python.ORDER_NONE) || "None";
  var sigma = block.getFieldValue("SIGMA") || "0.5";
  var resultVar = Blockly.Python.nameDB_.getDistinctName("out_im", Blockly.VARIABLE_CATEGORY_NAME);
  // code without im2im
  var code = `${resultVar} = gaussian(${image}, sigma=${sigma})`;
  // output
  Blockly.Python.definitions_["define_" + resultVar] = code;
  return [`${resultVar}`, Blockly.Python.ORDER_NONE];
};

With im2im

Blockly.Python.forBlock['gaussian_blur'] = function (block) {
  // inputs as above ...
  // code with im2im
  var convert_to = `in_im1 = im2im(${image}, 'skimage.before_gaussian')`;
  var operation = `e_gaussian_filtered = gaussian(in_im1.raw_image, sigma=${sigma})`;
  var convert_back = `${resultVar} = Image(e_gaussian_filtered, {**in_im1.metadata, 'image_data_type': 'float64(0to1)'})`;
  var code = `${convert_to}\n${operation}\n${convert_back}`;
  // outputs as above...
};

Note: Preset strings are used to instantiate an Image.

Note: If you want to display the conversion result, Please use raw_image of Image.

Note: For additional implementation examples, see the comparative_analysis/1/enhanced_VPL4IP.html and comparative_analysis/2/enhanced_VPL4IP.html .

Note: For detailed comparisons between visual programming systems (visual interface and execution) with and without im2im, refer to the following studies: , .

Document

For detailed APIs and usage, please refer to docs folder.

Contribution

We welcome all contributions to this project! If you have suggestions, feature requests, or want to contribute in any other way, please feel free to open an issue or submit a pull request. For detailed instructions on developing, building, and publishing this package, please refer to the README_DEV.

Cite

If you use this package in your research, please cite our paper:

@article{chen2025im2im,
  author    = {Fei Chen and Sunita Saha and Manuela Schuler and Philipp Slusallek and Tim Dahmen},
  title     = {im2im: Automatically Converting In-Memory Image Representations using A Knowledge Graph Approach},
  journal   = {Proc. ACM Program. Lang.},
  volume    = {9},
  number    = {OOPSLA2},
  pages     = {281:1--281:26},
  year      = {2025},
  month     = oct,
  doi       = {10.1145/3763059}
}

License

This project is licensed under the MIT License. See the LICENSE file for details.