Task with images

[arnaud-secondlayer]

I want to do some image processing in a task (I receive the image from a fastapi endpoint).
What is the best way to send images in a task? Today I convert images to a base64 string but I wonder if there is a more efficient way.

[s3rius]

More efficent way would be to send array of integers,

In [1]: a = b"test"

In [2]: list(a)
Out[2]: [116, 101, 115, 116]

In [3]: bytes(list(a))
Out[3]: b'test'

[danfimov]

If images are big, it probably worth to load them to S3 after recieving them in API endpoint. After that you can send link for this image to task as a parameter. Otherwise it can be not optimal to store big file in storage you use for tasks.

I saw a good solution for that problem in taskiq-aio-sqs, where author uses s3 for large taskiq messages automatically after some threshold. You can find code here.