EPIC: tag items to categorise and organize

[iteles]

As a person who believes that structure is key to organisation and trust in an app
I'd like to be able to tag my inputs with tags I make up myself
So that I can later have a better understanding of what is required from that capture at a glance and filter by the topics I am most interested in.

Tagging and projects are some of the features that I expect to most evolve as part of the app's user experience as they are the most critical but here is a starting point.

To be clear, I don't think that this will be useable as-is, but will certainly be a place to evolve from. We will have a much better UX for this within the next sprint but this is the starting point.

Acceptance Criteria

• When I edit an existing item in my list of captures items,
  I see a text field where I can input tags
• For stage 1, tags are input as a comma-separated list (a space required after each comma)
• A submit button is clicked to save them
• Each tag is checked against the saved tags (case-insensitive) and:
  • If a tag already exists, save the captures item against that tag
  • If a tag does not exist, create a new tag and save the captures item to it
  • Do not worry about leading and trailing spaces for now, this will not be the final UX MVP so we shouldn't waste time on fool-proofing for leading and trailing spaces yet - this version will only be used inside the dwyl team
• Once tags are saved, take human back to the list of captures items
• The list of tags should be displayed under the item.text
• each tag should be a clickable hyperlink
  • clicking/tapping on the tag (hyperlink) should filter the items in the current list to just those with that tag.

[SimonLab]

Thanks @iteles for these acceptance criteria.

Do not worry about leading and trailing spaces for now

I think it shouldn't be too complicated to trim the spaces on tags. Another question, do we want tags to be case insensitive? Might be good to avoid creating duplicated tags.
Singular vs plural tags might also be a case where duplicated can be created (eg idea vs ideas tag)?

[iteles]

@SimonLab All good questions!

On case sensitivity - definitely case-insensitive for now.

I wouldn't worry about the plurals yet because we will very closely get to an auto-complete solution to this where a person will type in a couple of letters and get the various suggestions given to them (which should deal with the plurals).

[nelsonic]

@iteles are you working on creating the UI for this story or am I'?
It's been in Sprint 1 milestone https://github.com/dwyl/app/milestone/3 for way too long ...
Can we either remove it from the milestone or provide some sort of update?

[nelsonic]

The UX of this is pretty important. This is a mini-project in itself that is worth brainstorming several UIs for to ensure that we have an API that can cater to all UI/UXs.
@iteles do you have time to sit down and sketch ideas with me e.g: tomorrow?

[nelsonic]

Next: Sketch the most basic UI for adding tags to items. ✍️

Happy to pick this up and do a quick sketch on paper or Figma so that we have something to work off. 💭
(unless anyone else want's to give it a go...)

[SimonLab]

I'm trying Excalidraw to quickly create UI of the mvp:

Excalidraw live session: https://excalidraw.com/#room=c7eca65eedec24628894,rm8LQPspjQO5OKF2gaxgsQ

It's really quick to create something simple, it doesn't have a lot of Figma features (simulate user actions) but I think it's enough and easier/quicker to use for what we want to represent.

I'll update the UI to represent the requirements define in #245 (comment)

My first idea based on the requirements above is to have a simple text input to add tags to a new item. We can then click on the list of tags of an item to edit them, the same way we edit the item itself.

The UI is simple but it would allow us to have the first version of this feature deploy rapidly and to iterate on it.

@nelsonic do you want to change this first UI version or should I try to implement it? Also should we moved this issue to https://github.com/dwyl/mvp/issues?

On the backend we can have a tags table and items-tags table to link tags to item, I'lll make sure that we can't create duplicate tags and that a same tag can't be added multiple time to an item.

[SimonLab]

Create backend logic

Currently we have the following tables:

We can now add the tags and items_tags table using migrations:

mix ecto.gen.migration add_tags

We can now add the code to create the tags table in the new migration file:

defmodule App.Repo.Migrations.AddTags do
  use Ecto.Migration

  def change do
    create table(:tags) do
      add(:text, :string)

      timestamps()
    end

    create(unique_index(:tags, ["lower(text)"]))
  end
end

The create(unique_index(:tags, ["lower(text)"]))and especially the lower function makes sure that we don't have any duplicates

mix ecto.gen.migration add_items_tags

defmodule App.Repo.Migrations.AddItemsTags do
  use Ecto.Migration

  def change do
    create table(:items_tags) do
      add(:item_id, references(:items, on_delete: :nothing))
      add(:tag_id, references(:tags, on_delete: :nothing))

      timestamps()
    end

    create(unique_index(:items_tags, [:item_id, :tag_id]))
  end
end

Adding a unique index to item_id and tag_id makes sure that we can't add the same tag multiple times to a same item

Now that we have the tables created with the migrations (make sure to run mix ecto.migrate) we can define the schemas

[nelsonic]

@SimonLab this proposed UI/UX is definitely the simplest for adding/removing tags in MVP:

I've tidied the Acceptance Criteria in the OP. 👍

[SimonLab]

Creating App.Tag schema and use the many_to_many function to link tags to items:

defmodule App.Tag do
  use Ecto.Schema
  import Ecto.Changeset
  alias App.{Item, ItemTag}

  schema "tags" do
    field :text, :string

    many_to_many(:items, Item, join_through: ItemTag)
    timestamps()
  end

  @doc false
  def changeset(tag, attrs) do
    tag
    |> cast(attrs, [:text])
    |> validate_required([:text])
  end
end

The ItemTag schema just define the links between item and tag:

defmodule App.ItemTag do
  use Ecto.Schema
  alias App.{Item, Tag}

  schema "items_tags" do
    belongs_to(:item, Item)
    belongs_to(:tag, Tag)

    timestamps()
  end
end

Now that we have our schemas defined, we need to make sure our changesets handle the unique constraint on the tag name and on the item - tag.

see

• https://hexdocs.pm/ecto/polymorphic-associations-with-many-to-many.html#polymorphism-with-many_to_many
• https://www.youtube.com/watch?v=pwTf887Mk6s&list=PLFhQVxlaKQElscjMvMmyMCaZ9mxf4XAw-&index=9&ab_channel=AlchemistCamp

[nelsonic]

@SimonLab not so sure about the unique on tags.text 🦄
It makes sense initially ... but if more than one person wants to use a given tag.text, 🏷️
they won't be able to unless we allow sharing tags between strangers. 💭

Basically, when someone is adding tags to their item, we should:

1. lookup to see if they already have the tag in their personal collection 🔍
2. if they don't already have the tag look it up in the generic tags list. 📝

   Note: never attempt to auto-complete from tags belonging to other people/orgs 🚫
   as it could "leak" confidential info. In fact tags.text should be Fields.Encrypted to ensure privacy. 🔐

3. Attempt to auto-correct the tag.text based on a dictionary of words but respect people's spelling if they reject the suggestion.
4. Save the new tag and associate it with the item. 🆕

[SimonLab]

My initial implementation was to make tags global to the all application, however I can easily change it to make tags unique to a user. For the first version I'm not using an autocomplete feature or dictionary to keep things simple, but we can add this later one if required

[nelsonic]

@SimonLab yeah having tag.text unique (Global) is fine for MVP
but no bueno for App where we want each person/team to have their own set of tags.

[nelsonic]

It basically means that in MVP we can use a fairly basic uniqueness check, 👍
but in the "real" App it needs a bit more logic and can't rely on Ecto to perform the checks. 💭

[SimonLab]

I've done more reading/testing with tags and items associations yesterday and I'm updating the documentation BUILDIT file:
dwyl/mvp@e4df192

I'll continue adding the doc and describe any blockers I encounter in this issue

[nelsonic]

Thanks @SimonLab makes sense. 👌🏻

[SimonLab]

All the backend logic for tags is now implemented.
I've added a simple text input to allow people to link tags to a new item:

Next I will look at displaying the tags on the existing item and I will need to complete the BUILDIT documentation.

[nelsonic]

@SimonLab sounds good. thanks for the update. 👌

[SimonLab]

I'm currently updating the following function to get the tags linked to the item:

  def items_with_timers(person_id \\ 0) do
    sql = """
    SELECT i.id, i.text, i.status, i.person_id, t.start, t.stop, t.id as timer_id FROM items i
    FULL JOIN timers as t ON t.item_id = i.id
    WHERE i.person_id = $1 AND i.status IS NOT NULL
    ORDER BY timer_id ASC;
    """

    Ecto.Adapters.SQL.query!(Repo, sql, [person_id])
    |> map_columns_to_values()
    |> accumulate_item_timers()
  end

I'm thinking of adding another join to get the tags.
I will have a look later on if we can simplify the query by using the schemas to access directly the timers and tags

[nelsonic]

@SimonLab yeah, for now adding a join to the query is probably going to be fastest. 💭
Very happy to refactor to use idiomatic ecto in future. 👍

[SimonLab]

I've updated the sql query to returns the list of tags for the items:

  def items_with_timers(person_id \\ 0) do
    sql = """
    SELECT i.id, i.text, i.status, i.person_id, t.start, t.stop, t.id as timer_id, tags.text as tag_text, tags.id as tag_id FROM items i
    FULL JOIN timers as t ON t.item_id = i.id
    FULL JOIN items_tags as it ON it.item_id = i.id
    FULL JOIN tags ON it.tag_id = tags.id
    WHERE i.person_id = $1 AND i.status IS NOT NULL
    ORDER BY timer_id ASC;
    """

    Ecto.Adapters.SQL.query!(Repo, sql, [person_id])
    |> map_columns_to_values()
    |> accumulate_item_timers()
  end

However I had to change the map_column_to_values function to use AtomicMap.convert(safe: false) with the safe option set to false. Ideally we want to avoid converting string to atom (atoms are not garbage collected and if enough string are converted to atoms it's possible the memory could fail).
The other issue is that I need to change the logic of the accumulate_item_timers function as it doesn't keep the list of tags.

So I want first to understand fully the timer logic so I don't break anything in the code:
https://github.com/dwyl/mvp/blob/0270bb5b610496197a5d28dd87bbb7412745663e/lib/app/item.ex#L186-L202

[SimonLab]

Displaying the tag names under each items:

The filter are clickable and filter the list of items matching the tag name
I need now to finish to update the BUILDIT documentation and the PR should be ready for review.
The only feature not implemented yet from the list of requirements on this issue, is:

When I edit an existing item in my list of captures items,I see a text field where I can input tags

This requires a bit more work on the UI/front end side. I think the tag system will be updated soon (see #287 ) and I don't wont to add to much more code to this PR. I'd like also to focus on updating the templates by using Phoenix components.

[nelsonic]

@SimonLab thanks for the great PR. 👌
I reviewed: dwyl/mvp#150 (review) and merged it. 🚀
Let's test this iteration and see what else is needed. 💭

[nelsonic]

@iteles this is ready for testing on: https://mvp.fly.dev/ 🙏