ImaginAIry 🤖🧠

Pythonic AI generation of images and videos

Features

• Image Generation: Create with SDXL, Openjourney, OpenDalle, and many others.
  • Generation Control: Exert detailed control over the generation process.
• Image Editing: Edit images with instructions.
• Image Upscaling: Add details to images.
• Video Generation: Turn images into videos.
• Image Captioning:

---

Installation

# on macOS, make sure rust is installed first
# be sure to use Python 3.10, Python 3.11 is not supported at the moment
pip install imaginairy

---

Image Generation

CLIPython

aimg imagine --seed 1 --model sdxl --size hd "wide shot of sun setting on a green valley"

from imaginairy.api.generate import imagine
from imaginairy.schema import ImaginePrompt

prompt = ImaginePrompt(
    prompt="wide shot of sun setting on a green valley", 
    seed=1, 
    model_weights="sdxl", 
    size="hd"
)

result = next(imagine(prompts=prompt))
result.img.save("sun_set_green_valley.png")

---

Image Generation Control

Guide the generation process by providing body poses, depth maps, canny edges, hed boundaries, normal maps, or even QR codes.

Body Pose Control

CLIPython

imagine --control-image assets/indiana.jpg  --control-mode openpose --caption-text openpose "photo of a polar bear"

from imaginairy.api.generate import imagine
from imaginairy.schema import ImaginePrompt, ControlInput, LazyLoadingImage

image = LazyLoadingImage(filepath="assets/indiana.jpg")
control_mode = ControlInput(mode="openpose", image=image)
prompt = ImaginePrompt(prompt="photo of a polar bear", control_inputs=[control_mode], seed=1)

result = next(imagine(prompts=prompt))
result.img.save("assets/indiana-pose-polar-bear.jpg")

Canny Edge Control

CLIPython

imagine --control-image assets/lena.png  --control-mode canny "photo of a woman with a hat looking at the camera"

from imaginairy.api.generate import imagine
from imaginairy.schema import ImaginePrompt, ControlInput, LazyLoadingImage

image = LazyLoadingImage(filepath="assets/lena.png")
control_mode = ControlInput(mode="canny", image=image)
prompt = ImaginePrompt(prompt="photo of a woman with a hat looking at the camera", control_inputs=[control_mode], seed=1)

result = next(imagine(prompts=prompt))
result.img.save("assets/lena-canny-generated.jpg")

HED Boundary Control

CLIPython

imagine --control-image dog.jpg  --control-mode hed  "photo of a dalmation"

from imaginairy.api.generate import imagine
from imaginairy.schema import ImaginePrompt, ControlInput, LazyLoadingImage

image = LazyLoadingImage(filepath="assets/000032_337692011_PLMS40_PS7.5_a_photo_of_a_dog.jpg")
control_mode = ControlInput(mode="hed", image=image)
prompt = ImaginePrompt(prompt="photo of a dalmation", control_inputs=[control_mode], seed=1)

result = next(imagine(prompts=prompt))
result.img.save("assets/dog-hed-boundary-dalmation.jpg")

Depth Map Control

CLIPython

imagine --control-image fancy-living.jpg  --control-mode depth  "a modern living room"

from imaginairy.api.generate import imagine
from imaginairy.schema import ImaginePrompt, ControlInput, LazyLoadingImage

image = LazyLoadingImage(filepath="assets/fancy-living.jpg")
control_mode = ControlInput(mode="depth", image=image)
prompt = ImaginePrompt(prompt="a modern living room", control_inputs=[control_mode], seed=1)

result = next(imagine(prompts=prompt))
result.img.save("assets/fancy-living-depth-generated.jpg")

Normal Map Control

CLIPython

imagine --control-image bird.jpg  --control-mode normal  "a bird"

from imaginairy.api.generate import imagine
from imaginairy.schema import ImaginePrompt, ControlInput, LazyLoadingImage

image = LazyLoadingImage(filepath="assets/013986_1_kdpmpp2m59_PS7.5_a_bluejay_[generated].jpg")
control_mode = ControlInput(mode="normal", image=image)
prompt = ImaginePrompt(prompt="a bird", control_inputs=[control_mode], seed=1)

result = next(imagine(prompts=prompt))
result.img.save("assets/bird-normal-generated.jpg")

Image Shuffle Control

Generates the image based on elements of the control image. Kind of similar to style transfer.

CLIPython

imagine --control-image pearl-girl.jpg  --control-mode shuffle  "a clown"

from imaginairy.api.generate import imagine
from imaginairy.schema import ImaginePrompt, ControlInput, LazyLoadingImage

image = LazyLoadingImage(filepath="assets/girl_with_a_pearl_earring.jpg")
control_mode = ControlInput(mode="shuffle", image=image)
prompt = ImaginePrompt(prompt="a clown", control_inputs=[control_mode], seed=1)

result = next(imagine(prompts=prompt))
result.img.save("assets/pearl_shuffle_clown_019331_1_kdpmpp2m15_PS7.5_img2img-0.0_a_clown.jpg")

The middle image is the "shuffled" input image

Editing Instructions Control

Similar to instructPix2Pix (below) but works with any SD 1.5 based model.

CLIPython

imagine --control-image pearl-girl.jpg  --control-mode edit --init-image-strength 0.01 --steps 30  --negative-prompt "" --model openjourney-v2 "make it anime" "make it at the beach" 

from imaginairy.api.generate import imagine
from imaginairy.schema import ImaginePrompt, ControlInput, LazyLoadingImage

image = LazyLoadingImage(filepath="assets/girl_with_a_pearl_earring.jpg")
control_mode = ControlInput(mode="edit", image=image)

prompts = [ImaginePrompt(prompt="make it anime", control_inputs=[control_mode], init_image_strength=0.01, steps=30, negative_prompt="", model_weights="openjourney-v2"),
            ImaginePrompt(prompt="make it at the beach", control_inputs=[control_mode], init_image_strength=0.01, steps=30, negative_prompt="", model_weights="openjourney-v2")]

imagine_iterator = imagine(prompts=prompts)

result = next(imagine_iterator)
result.img.save("assets/pearl_anime_019537_521829407_kdpmpp2m30_PS9.0_img2img-0.01_make_it_anime.jpg")

result = next(imagine_iterator)
result.img.save("assets/pearl_beach_019561_862735879_kdpmpp2m30_PS7.0_img2img-0.01_make_it_at_the_beach.jpg")

Add Details Control (upscaling/super-resolution)

Replaces existing details in an image. Good to use with --init-image-strength 0.2

CLIPython

imagine --control-image "assets/wishbone.jpg" --control-mode details "sharp focus, high-resolution" --init-image-strength 0.2 --steps 30 --size 2048x2048 

from imaginairy.api.generate import imagine
from imaginairy.schema import ImaginePrompt, ControlInput, LazyLoadingImage

image = LazyLoadingImage(filepath="assets/wishbone_headshot_badscale.jpg")
control_mode = ControlInput(mode="details", image=image)
prompt = ImaginePrompt(prompt="sharp focus, high-resolution", control_inputs=[control_mode], init_image_strength=0.2)

result = next(imagine(prompts=prompt))
result.img.save("assets/wishbone_headshot_details.jpg")

Image (re)Colorization (using brightness control)

Colorize black and white images or re-color existing images.

The generated colors will be applied back to the original image. You can either provide a caption or allow the tool to generate one for you.

CLIPython

aimg colorize pearl-girl.jpg --caption "photo of a woman"

from imaginairy.api.colorize import colorize_img
from PIL import Image, ImageEnhance, ImageStat

init_image = Image.open("assets/girl_with_a_pearl_earring.jpg")
image = colorize_img(img=init_image, caption="photo of a woman")
image.save("assets/pearl-colorized.jpg")

Image Upscaling

Upscale images easily.

CLIPython

aimg upscale assets/000206_856637805_PLMS40_PS7.5_colorful_smoke.jpg --upscale-model real-hat

from imaginairy.api.upscale import upscale

img = upscale(img="assets/000206_856637805_PLMS40_PS7.5_colorful_smoke.jpg")
img.save("colorful_smoke.upscaled.jpg")

➡️

Upscaling uses Spandrel to make it easy to use different upscaling models. You can view different integrated models by running aimg upscale --list-models, and then use it with --upscale-model <model-name>. Also accepts url's if you want to upscale an image with a different model. Control the new file format/location with --format.

Video Generation

CLIPython

aimg videogen --start-image assets/rocket-wide.png

from imaginairy.api.video_sample import generate_video

generate_video(input_path="assets/rocket-wide.png")