The algorithms used to render 3D CG graphics require scene-linear colors, and images intended for image-based lighting are typically supplied as HDR or OpenEXR images that are scene-linear already. However, the images might still require some color management.
If the primaries used to encode an image do not match the working space used for rendering, then the image should be converted using one or more of the transforms in the primaries/ directory. For example, if the image uses the Adobe RGB primaries and you want to render using the UHDTV (Rec. 2020) primaries, then you can apply LinearAdobeRGB_to_CIE-XYZ followed by CIE-XYZ_to_LinearUHDTV.