Engineering

26 July, 2022

How to support a list of uploads as input with Absinthe GraphQL

As you might guess, in our day-to-day, we write GraphQL queries and mutations for Phoenix applications using Absinthe to be able to create, read, update and delete records.

Nuno Marinho

Software Engineer

Here at Coletiv, we adopted Elixir in 2017. And to build a full-blown GraphQL service we use Absinthe, a GraphQL implementation for Elixir.

As you might guess, in our day-to-day, we write GraphQL queries and mutations for Phoenix applications using Absinthe to be able to create, read, update and delete records. In this context, the following feature was requested:

"add a group of photos/gallery when creating or editing one user”

A first (wrong) attempt

Without much thought (please don’t do this at home 😏), we immediately configured an AWS S3 bucket to host the images and changed the database accordingly by adding a column to the User table that would contain an array of images.

Initial User module

 defmodule GraphiQLImages.User do
  @moduledoc """
  Module Schema for `User`.
  """

  use GraphiQLImages.Schema
  import Ecto.Changeset

  schema "users" do
    field(:email, :string)
    field(:name, :string)
    field(:surname, :string)

    timestamps()
  end

  @email_regex ~r/^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,12}$/

  @required_fields [:email, :name, :surname]
  @optional_fields []

  def changeset(user, attrs) do
    user
    |> cast(attrs, @required_fields ++ @optional_fields)
    |> validate_required(@required_fields, message: "field_is_required")
    |> unique_constraint(:email, message: "email_has_been_taken")
    |> validate_format(:email, @email_regex, message: "invalid_email_format")
  end
end

Updated User module

defmodule GraphiQLImages.User do
  @moduledoc """
  Module Schema for `User`.
  """

  use GraphiQLImages.Schema
  import Ecto.Changeset

	alias GraphiQLImages.General.ImageEmbedded

  schema "users" do
    field(:email, :string)
    field(:name, :string)
    field(:surname, :string)

		embeds_many(:gallery, ImageEmbedded, on_replace: :delete)

    timestamps()
  end

  @email_regex ~r/^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,12}$/

  @required_fields [:email, :name, :surname]
  @optional_fields []

  def changeset(user, attrs) do
    user
    |> cast(attrs, @required_fields ++ @optional_fields)
    |> validate_required(@required_fields, message: "field_is_required")
    |> unique_constraint(:email, message: "email_has_been_taken")
    |> validate_format(:email, @email_regex, message: "invalid_email_format")
	  |> cast_embed(:gallery)
	end
end

Now that we have the S3 bucket configured and the database schema adjusted, the next step is to adjust the mutation. We need to allow the API users to send the array of upload alongside the remaining user data (e.g.: email and name).

Initial User types

defmodule GraphiQLImagesGraphQL.User.Types do
  @moduledoc """
  Users Types
  """
  use Absinthe.Schema.Notation
  import Absinthe.Resolution.Helpers, only: [dataloader: 1]

  alias GraphiQLImagesGraphQL.User.Resolver

  ##################
  # Inputs Objects #
  ##################

  @desc "The user input object"
  input_object :user_input do
    field(:email, non_null(:string))
    field(:name, non_null(:string))
    field(:surname, non_null(:string))
  end

  @desc "The update user input object"
  input_object :update_user_input do
    field(:email, :string)
    field(:name, :string)
    field(:surname, :string)
  end

  ###############
  #   Objects   #
  ###############

  @desc "The user object"
  object :user do
    ...
  end

  ############
  # Queries  #
  ############

  object :user_queries do
    ...
  end

  ##############
  # Mutations  #
  ##############

  object :user_mutations do
    @desc "Create new user"
    field(:user_create, :user) do
      arg(:input, non_null(:user_input))
      resolve(&Resolver.user_create/2)
    end

    @desc "Edit an user"
    field(:user_update, :user) do
      arg(:id, non_null(:id))
      arg(:input, non_null(:update_user_input))
      resolve(&Resolver.user_update/2)
    end
  end
end

Update User types

defmodule GraphiQLImagesGraphQL.User.Types do
  @moduledoc """
  Users Types
  """
  use Absinthe.Schema.Notation
  import Absinthe.Resolution.Helpers, only: [dataloader: 1]

  alias GraphiQLImagesGraphQL.User.Resolver

  ##################
  # Inputs Objects #
  ##################

  @desc "The user input object"
  input_object :user_input do
    field(:email, non_null(:string))
    field(:name, non_null(:string))
    field(:surname, non_null(:string))
    field(:gallery, non_null(list_of(non_null(:upload))))
  end

  @desc "The update user input object"
  input_object :update_user_input do
    field(:email, :string)
    field(:name, :string)
    field(:surname, :string)
    field(:gallery, list_of(non_null(:upload)))
  end

  ###############
  #   Objects   #
  ###############

  @desc "The user object"
  object :user do
    ...
  end

  ############
  # Queries  #
  ############

  object :user_queries do
    ...
  end

  ##############
  # Mutations  #
  ##############

  object :user_mutations do
    @desc "Create new user"
    field(:user_create, :user) do
      arg(:input, non_null(:user_input))
      resolve(&Resolver.user_create/2)
    end

    @desc "Edit an user"
    field(:user_update, :user) do
      arg(:id, non_null(:id))
      arg(:input, non_null(:update_user_input))
      resolve(&Resolver.user_update/2)
    end
  end
end

Problem

When we add list_of(:upload)

field(:gallery, non_null(non_null(:upload))))

⚠️We found a problem⚠️ Absinthe doesn't allow the use of list_of(:upload)

https://github.com/absinthe-graphql/absinthe_plug/pull/201

Since this solution relies on a new GraphQL convention that doesn't exist elsewhere, I don't think we can adopt this.

Perhaps you could submit multiple mutations in a single GraphQL operation, one per file that you wish to upload? This is already possible and part of the GraphQL spec...

Closing this ticket hoping that this solution is workable!

Putting things in these terms, it is quite impossible to continue with the solution used so far.

A second attempt that actually works 🥳:

We got back to the drawing board and decided to create an image table with a foreign key to the user's table. And enable the addition of images separately from the creation of a user, providing a separate mutation just for adding images.

GalleryImage module

defmodule GraphiQLImages.User.GalleryImage do
  @moduledoc """
  The Gallery Image schema module
  """
  use GraphiQLImages.Schema
  import Ecto.Changeset

  alias GraphiQLImages.General.Image.ImageEmbedded
  alias GraphiQLImages.User

  schema "user_gallery_images" do
    embeds_one :gallery_image, ImageEmbedded, on_replace: :delete
    belongs_to(:user, User)

    timestamps()
  end

  @required_fields []
  @optional_fields [:user_id]

  def changeset(gallery_image, params) do
    gallery_image
    |> cast(params, @required_fields ++ @optional_fields)
    |> foreign_key_constraint(:user_id, message: "invalid_identifier")
    |> cast_embed(:gallery_image)
  end
end

After we made all the changes in the user’s schema, we now need to update the mutations and types.
The solution was to allow the upload of an array of IDs when creating a user. These IDs are obtained by first uploading the images via the User_gallery_image_create mutation, which are then used/passed in the create user mutation.
To sum it up, we first execute the upload image mutation that returns the id that we later use in the create user mutation.

GalleryImage types

defmodule GraphiQLImagesGraphQl.User.GalleryImage.Types do
  @moduledoc """
  GalleryImage Types
  """
  use Absinthe.Schema.Notation

  alias GraphiQLImagesGraphQl.General.Resolver, as: GeneralResolver
  alias GraphiQLImagesGraphQl.User.GalleryImage.Resolver, as: UserGalleryImageResolver

  ###############
  #   Objects   #
  ###############

  @desc "The user gallery image object"
  object(:user_gallery_image) do
    field(:id, non_null(:id))

    field(:gallery_image, non_null(:file_image)) do
      resolve(&GeneralResolver.gallery_image/3)
    end

    import_fields(:timestamps)
  end

  ##############
  # Mutations  #
  ##############

  object(:user_gallery_image_mutations) do
    @desc "Create a new user gallery image"
    field(:user_gallery_image_create, :user_gallery_image) do
      arg(:gallery_image, non_null(:upload))

      resolve(&UserGalleryImageResolver.user_gallery_image_create/2)
    end
  end
end

User types

defmodule GraphiQLImagesGraphQL.User.Types do
  @moduledoc """
  Users Types
  """
  use Absinthe.Schema.Notation
  import Absinthe.Resolution.Helpers, only: [dataloader: 1]

  alias GraphiQLImagesGraphQL.User.Resolver

  ##################
  # Inputs Objects #
  ##################

  @desc "The user input object"
  input_object :user_input do
    field(:email, non_null(:string))
    field(:name, non_null(:string))
    field(:surname, non_null(:string))
    field(:gallery, list_of(non_null(:id)))
  end

  @desc "The update user input object"
  input_object :update_user_input do
    field(:email, :string)
    field(:name, :string)
    field(:surname, :string)
    field(:gallery, list_of(non_null(:id)))
  end

  ###############
  #   Objects   #
  ###############

  @desc "The user object"
  object :user do
    field(:id, non_null(:id))
    field(:email, non_null(:string))
    field(:name, non_null(:string))
    field(:surname, non_null(:string))

    field(:gallery, list_of(:user_gallery_image)) do
      resolve(dataloader(Repo))
    end

    import_fields(:timestamps)
  end

  ############
  # Queries  #
  ############

  object :user_queries do
    @desc "Get an user by `id`"
    field :user, :user do
      arg(:id, non_null(:id))
      resolve(&Resolver.user/3)
    end
  end

  ##############
  # Mutations  #
  ##############

  object :user_mutations do
    @desc "Create new user"
    field(:user_create, :user) do
      arg(:input, non_null(:user_input))
      resolve(&Resolver.user_create/2)
    end

    @desc "Edit an user"
    field(:user_update, :user) do
      arg(:id, non_null(:id))
      arg(:input, non_null(:update_user_input))
      resolve(&Resolver.user_update/2)
    end
  end
end

Prevent the existence of unused images

The process described before has the problem of possibly creating/storing images that are never used. For example, a user can upload several images but reloads the browser before actually creating the user.
To prevent this, we created a GenServer that, every day, checks for images that are not in use and deletes them from the S3 bucket.

ImageCleanWorker process

defmodule GraphiQLImages.User.GalleryImage.Process.ImageCleanWorker do
  @moduledoc """
  Module worker to clean unused images (image inserted at least 2 days ago) from the database.
  """
  use GenServer, restart: :transient

  alias GraphiQLImages.General.Clock
  alias GraphiQLImages.User.GalleryImage.Query, as: GalleryImageQuery
  alias GraphiQLImages.User.GalleryImages

  def start_link(arg) do
    GenServer.start_link(__MODULE__, arg, name: :image_clean_worker)
  end

  def init(state) do
    schedule()
    {:ok, state}
  end

  def handle_info(:run_image_clean_worker, state) do
    run_image_clean_worker()
    schedule()

    {:noreply, state}
  end

  # Clean unused images (image inserted at least 2 days ago) from the database and S3
  defp run_image_clean_worker do
    GalleryImageQuery.filter_by_nil_user()
    |> GalleryImageQuery.filter_by_inserted_at(Timex.shift(Clock.now(), days: -2))
    |> GalleryImages.delete()
  end

  defp schedule,
    do:
      Process.send_after(
        self(),
        :run_image_clean_worker,
        next_day_schedule_time()
      )

  defp next_day_schedule_time(schedule_hours \\ 4) do
    # ensure the next run hour is exactly the @email_sending_hour [hour]
    next_hour = 24 - DateTime.utc_now().hour + schedule_hours
    # ensure the minute is exactly 0 [ms * sec]
    next_minutes = 1000 * 60 * DateTime.utc_now().minute

    # [ms * sec * minute]
    1000 * 60 * 60 * next_hour - next_minutes
  end
end

This second attempt is available in this repository.

That’s a wrap

Hope the article helps you support the upload of multiple images on a form.

Even though our final solution is not perfect. As it requires the user’s API to do a multi-step form submission when creating a user: first upload the images to the API and then submit the form data with the IDs received in the previous step.

Our skilled frontend team managed to turn the cons into pros by immediately uploading the images, while the user is still filling in the remaining fields, making the whole process faster for the user.

Feel free to check out our repository and create a pull request with a different solution, we are always eager to look into better solutions!

As a final note, you might have noticed the reference to Balu in the code snippets. Balu is an internal project that we have been building to help animal associations list sheltered animals for adoption. There are plenty of other features we want to add, so stay tuned!

Elixir

Absinthe

GraphQL

Phoneix