05_columns.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to DataFrames\n",
    "**[Bogumił Kamiński](http://bogumilkaminski.pl/about/), December 12, 2021**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "using DataFrames"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Manipulating columns of a `DataFrame`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Renaming columns\n",
    "\n",
    "Let's start with a `DataFrame` of `Bool`s that has default column names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 4 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th></tr><tr><th></th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>1</td><td>0</td><td>1</td></tr><tr><th>2</th><td>1</td><td>1</td><td>1</td><td>1</td></tr><tr><th>3</th><td>1</td><td>0</td><td>1</td><td>0</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1 & x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Bool & Bool & Bool & Bool\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 1 & 0 & 1 \\\\\n",
       "\t2 & 1 & 1 & 1 & 1 \\\\\n",
       "\t3 & 1 & 0 & 1 & 0 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1   \u001b[0m\u001b[1m x2    \u001b[0m\u001b[1m x3    \u001b[0m\u001b[1m x4    \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Bool \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\n",
       "─────┼───────────────────────────\n",
       "   1 │ true   true  false   true\n",
       "   2 │ true   true   true   true\n",
       "   3 │ true  false   true  false"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame(rand(Bool, 3, 4), :auto)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With `rename`, we create new `DataFrame`; here we rename the column `:x1` to `:A`. (`rename` also accepts collections of Pairs.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 4 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>x2</th><th>x3</th><th>x4</th></tr><tr><th></th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>1</td><td>0</td><td>1</td></tr><tr><th>2</th><td>1</td><td>1</td><td>1</td><td>1</td></tr><tr><th>3</th><td>1</td><td>0</td><td>1</td><td>0</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& A & x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Bool & Bool & Bool & Bool\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 1 & 0 & 1 \\\\\n",
       "\t2 & 1 & 1 & 1 & 1 \\\\\n",
       "\t3 & 1 & 0 & 1 & 0 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m A    \u001b[0m\u001b[1m x2    \u001b[0m\u001b[1m x3    \u001b[0m\u001b[1m x4    \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Bool \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\n",
       "─────┼───────────────────────────\n",
       "   1 │ true   true  false   true\n",
       "   2 │ true   true   true   true\n",
       "   3 │ true  false   true  false"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rename(x, :x1 => :A)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With `rename!` we do an in place transformation. \n",
    "\n",
    "This time we've applied a function to every column name (note that the function gets a column names as a string)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 4 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1x1</th><th>x2x2</th><th>x3x3</th><th>x4x4</th></tr><tr><th></th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>1</td><td>0</td><td>1</td></tr><tr><th>2</th><td>1</td><td>1</td><td>1</td><td>1</td></tr><tr><th>3</th><td>1</td><td>0</td><td>1</td><td>0</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1x1 & x2x2 & x3x3 & x4x4\\\\\n",
       "\t\\hline\n",
       "\t& Bool & Bool & Bool & Bool\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 1 & 0 & 1 \\\\\n",
       "\t2 & 1 & 1 & 1 & 1 \\\\\n",
       "\t3 & 1 & 0 & 1 & 0 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1x1 \u001b[0m\u001b[1m x2x2  \u001b[0m\u001b[1m x3x3  \u001b[0m\u001b[1m x4x4  \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Bool \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\n",
       "─────┼───────────────────────────\n",
       "   1 │ true   true  false   true\n",
       "   2 │ true   true   true   true\n",
       "   3 │ true  false   true  false"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rename!(c -> c^2, x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also change the name of a particular column without knowing the original.\n",
    "\n",
    "Here we change the name of the third column, creating a new `DataFrame`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 4 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1x1</th><th>x2x2</th><th>third</th><th>x4x4</th></tr><tr><th></th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>1</td><td>0</td><td>1</td></tr><tr><th>2</th><td>1</td><td>1</td><td>1</td><td>1</td></tr><tr><th>3</th><td>1</td><td>0</td><td>1</td><td>0</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1x1 & x2x2 & third & x4x4\\\\\n",
       "\t\\hline\n",
       "\t& Bool & Bool & Bool & Bool\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 1 & 0 & 1 \\\\\n",
       "\t2 & 1 & 1 & 1 & 1 \\\\\n",
       "\t3 & 1 & 0 & 1 & 0 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1x1 \u001b[0m\u001b[1m x2x2  \u001b[0m\u001b[1m third \u001b[0m\u001b[1m x4x4  \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Bool \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\n",
       "─────┼───────────────────────────\n",
       "   1 │ true   true  false   true\n",
       "   2 │ true   true   true   true\n",
       "   3 │ true  false   true  false"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rename(x, 3 => :third)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we pass a vector of names to `rename!`, we can change the names of all variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 4 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th><th>c</th><th>d</th></tr><tr><th></th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>1</td><td>0</td><td>1</td></tr><tr><th>2</th><td>1</td><td>1</td><td>1</td><td>1</td></tr><tr><th>3</th><td>1</td><td>0</td><td>1</td><td>0</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& a & b & c & d\\\\\n",
       "\t\\hline\n",
       "\t& Bool & Bool & Bool & Bool\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 1 & 0 & 1 \\\\\n",
       "\t2 & 1 & 1 & 1 & 1 \\\\\n",
       "\t3 & 1 & 0 & 1 & 0 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a    \u001b[0m\u001b[1m b     \u001b[0m\u001b[1m c     \u001b[0m\u001b[1m d     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Bool \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\n",
       "─────┼───────────────────────────\n",
       "   1 │ true   true  false   true\n",
       "   2 │ true   true   true   true\n",
       "   3 │ true  false   true  false"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rename!(x, [:a, :b, :c, :d])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In all the above examples you could have used strings instead of symbols, e.g."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 4 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th><th>c</th><th>d</th></tr><tr><th></th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>1</td><td>0</td><td>1</td></tr><tr><th>2</th><td>1</td><td>1</td><td>1</td><td>1</td></tr><tr><th>3</th><td>1</td><td>0</td><td>1</td><td>0</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& a & b & c & d\\\\\n",
       "\t\\hline\n",
       "\t& Bool & Bool & Bool & Bool\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 1 & 0 & 1 \\\\\n",
       "\t2 & 1 & 1 & 1 & 1 \\\\\n",
       "\t3 & 1 & 0 & 1 & 0 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a    \u001b[0m\u001b[1m b     \u001b[0m\u001b[1m c     \u001b[0m\u001b[1m d     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Bool \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\n",
       "─────┼───────────────────────────\n",
       "   1 │ true   true  false   true\n",
       "   2 │ true   true   true   true\n",
       "   3 │ true  false   true  false"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rename!(x, string.('a':'d'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`rename!` allows for circular renaming of columns, e.g.:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 4 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th><th>c</th><th>d</th></tr><tr><th></th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>1</td><td>0</td><td>1</td></tr><tr><th>2</th><td>1</td><td>1</td><td>1</td><td>1</td></tr><tr><th>3</th><td>1</td><td>0</td><td>1</td><td>0</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& a & b & c & d\\\\\n",
       "\t\\hline\n",
       "\t& Bool & Bool & Bool & Bool\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 1 & 0 & 1 \\\\\n",
       "\t2 & 1 & 1 & 1 & 1 \\\\\n",
       "\t3 & 1 & 0 & 1 & 0 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a    \u001b[0m\u001b[1m b     \u001b[0m\u001b[1m c     \u001b[0m\u001b[1m d     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Bool \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\n",
       "─────┼───────────────────────────\n",
       "   1 │ true   true  false   true\n",
       "   2 │ true   true   true   true\n",
       "   3 │ true  false   true  false"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 4 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>d</th><th>b</th><th>c</th><th>a</th></tr><tr><th></th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>1</td><td>0</td><td>1</td></tr><tr><th>2</th><td>1</td><td>1</td><td>1</td><td>1</td></tr><tr><th>3</th><td>1</td><td>0</td><td>1</td><td>0</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& d & b & c & a\\\\\n",
       "\t\\hline\n",
       "\t& Bool & Bool & Bool & Bool\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 1 & 0 & 1 \\\\\n",
       "\t2 & 1 & 1 & 1 & 1 \\\\\n",
       "\t3 & 1 & 0 & 1 & 0 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m d    \u001b[0m\u001b[1m b     \u001b[0m\u001b[1m c     \u001b[0m\u001b[1m a     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Bool \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\n",
       "─────┼───────────────────────────\n",
       "   1 │ true   true  false   true\n",
       "   2 │ true   true   true   true\n",
       "   3 │ true  false   true  false"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rename!(x, \"a\"=>\"d\", \"d\"=>\"a\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We get an error when we try to provide duplicate names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "ename": "LoadError",
     "evalue": "ArgumentError: Duplicate variable names: :a. Pass makeunique=true to make them unique using a suffix automatically.",
     "output_type": "error",
     "traceback": [
      "ArgumentError: Duplicate variable names: :a. Pass makeunique=true to make them unique using a suffix automatically.",
      "",
      "Stacktrace:",
      " [1] rename!(x::DataFrames.Index, nms::Vector{Symbol}; makeunique::Bool)",
      "   @ DataFrames C:\\Users\\bogum\\.julia\\packages\\DataFrames\\ORSVA\\src\\other\\index.jl:51",
      " [2] #rename!#55",
      "   @ C:\\Users\\bogum\\.julia\\packages\\DataFrames\\ORSVA\\src\\abstractdataframe\\abstractdataframe.jl:199 [inlined]",
      " [3] rename(df::DataFrame, vals::Vector{Symbol}; makeunique::Bool)",
      "   @ DataFrames C:\\Users\\bogum\\.julia\\packages\\DataFrames\\ORSVA\\src\\abstractdataframe\\abstractdataframe.jl:323",
      " [4] rename(df::DataFrame, vals::Vector{Symbol})",
      "   @ DataFrames C:\\Users\\bogum\\.julia\\packages\\DataFrames\\ORSVA\\src\\abstractdataframe\\abstractdataframe.jl:323",
      " [5] top-level scope",
      "   @ In[10]:1",
      " [6] eval",
      "   @ .\\boot.jl:373 [inlined]",
      " [7] include_string(mapexpr::typeof(REPL.softscope), mod::Module, code::String, filename::String)",
      "   @ Base .\\loading.jl:1196"
     ]
    }
   ],
   "source": [
    "rename(x, fill(:a, 4))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " unless we pass `makeunique=true`, which allows us to handle duplicates in passed names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 4 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>a_1</th><th>a_2</th><th>a_3</th></tr><tr><th></th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>1</td><td>0</td><td>1</td></tr><tr><th>2</th><td>1</td><td>1</td><td>1</td><td>1</td></tr><tr><th>3</th><td>1</td><td>0</td><td>1</td><td>0</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& a & a\\_1 & a\\_2 & a\\_3\\\\\n",
       "\t\\hline\n",
       "\t& Bool & Bool & Bool & Bool\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 1 & 0 & 1 \\\\\n",
       "\t2 & 1 & 1 & 1 & 1 \\\\\n",
       "\t3 & 1 & 0 & 1 & 0 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a    \u001b[0m\u001b[1m a_1   \u001b[0m\u001b[1m a_2   \u001b[0m\u001b[1m a_3   \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Bool \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\n",
       "─────┼───────────────────────────\n",
       "   1 │ true   true  false   true\n",
       "   2 │ true   true   true   true\n",
       "   3 │ true  false   true  false"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rename(x, fill(:a, 4), makeunique=true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Reordering columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can reorder the `names(x)` vector as needed, creating a new `DataFrame`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 4 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>d</th><th>b</th><th>c</th><th>a</th></tr><tr><th></th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>1</td><td>0</td><td>1</td></tr><tr><th>2</th><td>1</td><td>1</td><td>1</td><td>1</td></tr><tr><th>3</th><td>1</td><td>0</td><td>1</td><td>0</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& d & b & c & a\\\\\n",
       "\t\\hline\n",
       "\t& Bool & Bool & Bool & Bool\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 1 & 0 & 1 \\\\\n",
       "\t2 & 1 & 1 & 1 & 1 \\\\\n",
       "\t3 & 1 & 0 & 1 & 0 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m d    \u001b[0m\u001b[1m b     \u001b[0m\u001b[1m c     \u001b[0m\u001b[1m a     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Bool \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\n",
       "─────┼───────────────────────────\n",
       "   1 │ true   true  false   true\n",
       "   2 │ true   true   true   true\n",
       "   3 │ true  false   true  false"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "using Random\n",
    "Random.seed!(1234)\n",
    "x[:, shuffle(names(x))]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Also `select!` can be used to achieve this in place (or `select` to perform a copy):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 4 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>d</th><th>b</th><th>c</th><th>a</th></tr><tr><th></th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>1</td><td>0</td><td>1</td></tr><tr><th>2</th><td>1</td><td>1</td><td>1</td><td>1</td></tr><tr><th>3</th><td>1</td><td>0</td><td>1</td><td>0</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& d & b & c & a\\\\\n",
       "\t\\hline\n",
       "\t& Bool & Bool & Bool & Bool\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 1 & 0 & 1 \\\\\n",
       "\t2 & 1 & 1 & 1 & 1 \\\\\n",
       "\t3 & 1 & 0 & 1 & 0 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m d    \u001b[0m\u001b[1m b     \u001b[0m\u001b[1m c     \u001b[0m\u001b[1m a     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Bool \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\n",
       "─────┼───────────────────────────\n",
       "   1 │ true   true  false   true\n",
       "   2 │ true   true   true   true\n",
       "   3 │ true  false   true  false"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 4 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>c</th><th>b</th><th>d</th></tr><tr><th></th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th><th title=\"Bool\">Bool</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>0</td><td>1</td><td>1</td></tr><tr><th>2</th><td>1</td><td>1</td><td>1</td><td>1</td></tr><tr><th>3</th><td>0</td><td>1</td><td>0</td><td>1</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& a & c & b & d\\\\\n",
       "\t\\hline\n",
       "\t& Bool & Bool & Bool & Bool\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 0 & 1 & 1 \\\\\n",
       "\t2 & 1 & 1 & 1 & 1 \\\\\n",
       "\t3 & 0 & 1 & 0 & 1 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a     \u001b[0m\u001b[1m c     \u001b[0m\u001b[1m b     \u001b[0m\u001b[1m d    \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool \u001b[0m\n",
       "─────┼───────────────────────────\n",
       "   1 │  true  false   true  true\n",
       "   2 │  true   true   true  true\n",
       "   3 │ false   true  false  true"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "select!(x, 4:-1:1);\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Merging/adding columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 4 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1 & x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) \\\\\n",
       "\t2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) \\\\\n",
       "\t3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)  (1, 3)  (1, 4)\n",
       "   2 │ (2, 1)  (2, 2)  (2, 3)  (2, 4)\n",
       "   3 │ (3, 1)  (3, 2)  (3, 3)  (3, 4)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame([(i,j) for i in 1:3, j in 1:4], :auto)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With `hcat` we can merge two `DataFrame`s. Also [x y] syntax is supported but only when DataFrames have unique column names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 8 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>x1_1</th><th>x2_1</th><th>x3_1</th><th>x4_1</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccccccc}\n",
       "\t& x1 & x2 & x3 & x4 & x1\\_1 & x2\\_1 & x3\\_1 & x4\\_1\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple… & Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) & (1, 1) & (1, 2) & (1, 3) & (1, 4) \\\\\n",
       "\t2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) & (2, 1) & (2, 2) & (2, 3) & (2, 4) \\\\\n",
       "\t3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) & (3, 1) & (3, 2) & (3, 3) & (3, 4) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×8 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x1_1   \u001b[0m\u001b[1m x2_1   \u001b[0m\u001b[1m x3_1   \u001b[0m\u001b[1m x4_1   \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)  (1, 3)  (1, 4)  (1, 1)  (1, 2)  (1, 3)  (1, 4)\n",
       "   2 │ (2, 1)  (2, 2)  (2, 3)  (2, 4)  (2, 1)  (2, 2)  (2, 3)  (2, 4)\n",
       "   3 │ (3, 1)  (3, 2)  (3, 3)  (3, 4)  (3, 1)  (3, 2)  (3, 3)  (3, 4)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hcat(x, x, makeunique=true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can append a vector to a data frame with the following syntax:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 5 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>A</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Int64\">Int64</th></tr></thead><tbody><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>1</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>2</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>3</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& x1 & x2 & x3 & x4 & A\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple… & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) & 1 \\\\\n",
       "\t2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) & 2 \\\\\n",
       "\t3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) & 3 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m A     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼───────────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)  (1, 3)  (1, 4)      1\n",
       "   2 │ (2, 1)  (2, 2)  (2, 3)  (2, 4)      2\n",
       "   3 │ (3, 1)  (3, 2)  (3, 3)  (3, 4)      3"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = [x DataFrame(A=[1,2,3])]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we do the same but add column `:A` to the front."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 5 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>x1</th><th>x2</th><th>x3</th><th>x4</th></tr><tr><th></th><th title=\"Int64\">Int64</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td></tr><tr><th>2</th><td>2</td><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td></tr><tr><th>3</th><td>3</td><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& A & x1 & x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) \\\\\n",
       "\t2 & 2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) \\\\\n",
       "\t3 & 3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m A     \u001b[0m\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼───────────────────────────────────────\n",
       "   1 │     1  (1, 1)  (1, 2)  (1, 3)  (1, 4)\n",
       "   2 │     2  (2, 1)  (2, 2)  (2, 3)  (2, 4)\n",
       "   3 │     3  (3, 1)  (3, 2)  (3, 3)  (3, 4)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = [DataFrame(A=[1,2,3]) x]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A column can also be added in the middle. Here a brute-force method is used and a new `DataFrame` is created."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  4.200 μs (85 allocations: 6.59 KiB)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 5 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>A</th><th>x3</th><th>x4</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Int64\">Int64</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>1</td><td>(1, 3)</td><td>(1, 4)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>2</td><td>(2, 3)</td><td>(2, 4)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>3</td><td>(3, 3)</td><td>(3, 4)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& x1 & x2 & A & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Int64 & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & 1 & (1, 3) & (1, 4) \\\\\n",
       "\t2 & (2, 1) & (2, 2) & 2 & (2, 3) & (2, 4) \\\\\n",
       "\t3 & (3, 1) & (3, 2) & 3 & (3, 3) & (3, 4) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m A     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼───────────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)      1  (1, 3)  (1, 4)\n",
       "   2 │ (2, 1)  (2, 2)      2  (2, 3)  (2, 4)\n",
       "   3 │ (3, 1)  (3, 2)      3  (3, 3)  (3, 4)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "using BenchmarkTools\n",
    "@btime [$x[!, 1:2] DataFrame(A=[1,2,3]) $x[!, 3:4]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We could also do this with a specialized in place method `insertcols!`. Let's add `:newcol` to the `DataFrame` `y`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 6 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>newcol</th><th>x1</th><th>x2</th><th>x3</th><th>x4</th></tr><tr><th></th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>1</td><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td></tr><tr><th>2</th><td>2</td><td>2</td><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td></tr><tr><th>3</th><td>3</td><td>3</td><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccccc}\n",
       "\t& A & newcol & x1 & x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) \\\\\n",
       "\t2 & 2 & 2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) \\\\\n",
       "\t3 & 3 & 3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×6 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m A     \u001b[0m\u001b[1m newcol \u001b[0m\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64  \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼───────────────────────────────────────────────\n",
       "   1 │     1       1  (1, 1)  (1, 2)  (1, 3)  (1, 4)\n",
       "   2 │     2       2  (2, 1)  (2, 2)  (2, 3)  (2, 4)\n",
       "   3 │     3       3  (3, 1)  (3, 2)  (3, 3)  (3, 4)"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "insertcols!(y, 2, \"newcol\" => [1,2,3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to insert the same column name several times `makeunique=true` is needed as usual."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 7 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>newcol_1</th><th>newcol</th><th>x1</th><th>x2</th><th>x3</th><th>x4</th></tr><tr><th></th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>1</td><td>1</td><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td></tr><tr><th>2</th><td>2</td><td>2</td><td>2</td><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td></tr><tr><th>3</th><td>3</td><td>3</td><td>3</td><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccccc}\n",
       "\t& A & newcol\\_1 & newcol & x1 & x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64 & Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 1 & 1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) \\\\\n",
       "\t2 & 2 & 2 & 2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) \\\\\n",
       "\t3 & 3 & 3 & 3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×7 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m A     \u001b[0m\u001b[1m newcol_1 \u001b[0m\u001b[1m newcol \u001b[0m\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64    \u001b[0m\u001b[90m Int64  \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼─────────────────────────────────────────────────────────\n",
       "   1 │     1         1       1  (1, 1)  (1, 2)  (1, 3)  (1, 4)\n",
       "   2 │     2         2       2  (2, 1)  (2, 2)  (2, 3)  (2, 4)\n",
       "   3 │     3         3       3  (3, 1)  (3, 2)  (3, 3)  (3, 4)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "insertcols!(y, 2, :newcol => [1,2,3], makeunique=true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see how much faster it is to insert a column with `insertcols!` than with `hcat` using `@btime` (note that we use here a `Pair` notation as an example)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  1.260 μs (19 allocations: 1.81 KiB)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 5 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>A</th><th>x3</th><th>x4</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Int64\">Int64</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>1</td><td>(1, 3)</td><td>(1, 4)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>2</td><td>(2, 3)</td><td>(2, 4)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>3</td><td>(3, 3)</td><td>(3, 4)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& x1 & x2 & A & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Int64 & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & 1 & (1, 3) & (1, 4) \\\\\n",
       "\t2 & (2, 1) & (2, 2) & 2 & (2, 3) & (2, 4) \\\\\n",
       "\t3 & (3, 1) & (3, 2) & 3 & (3, 3) & (3, 4) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m A     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼───────────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)      1  (1, 3)  (1, 4)\n",
       "   2 │ (2, 1)  (2, 2)      2  (2, 3)  (2, 4)\n",
       "   3 │ (3, 1)  (3, 2)      3  (3, 3)  (3, 4)"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "@btime insertcols!(copy($x), 3, :A => [1,2,3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's use `insertcols!` to append a column in place (note that we dropped the index at which we insert the column)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 5 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>A</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Int64\">Int64</th></tr></thead><tbody><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>1</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>2</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>3</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& x1 & x2 & x3 & x4 & A\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple… & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) & 1 \\\\\n",
       "\t2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) & 2 \\\\\n",
       "\t3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) & 3 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m A     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼───────────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)  (1, 3)  (1, 4)      1\n",
       "   2 │ (2, 1)  (2, 2)  (2, 3)  (2, 4)      2\n",
       "   3 │ (3, 1)  (3, 2)  (3, 3)  (3, 4)      3"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "insertcols!(x, :A => [1,2,3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and to in place prepend a column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 6 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>B</th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>A</th></tr><tr><th></th><th title=\"Int64\">Int64</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Int64\">Int64</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>1</td></tr><tr><th>2</th><td>2</td><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>2</td></tr><tr><th>3</th><td>3</td><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>3</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccccc}\n",
       "\t& B & x1 & x2 & x3 & x4 & A\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Tuple… & Tuple… & Tuple… & Tuple… & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) & 1 \\\\\n",
       "\t2 & 2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) & 2 \\\\\n",
       "\t3 & 3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) & 3 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×6 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m B     \u001b[0m\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m A     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼──────────────────────────────────────────────\n",
       "   1 │     1  (1, 1)  (1, 2)  (1, 3)  (1, 4)      1\n",
       "   2 │     2  (2, 1)  (2, 2)  (2, 3)  (2, 4)      2\n",
       "   3 │     3  (3, 1)  (3, 2)  (3, 3)  (3, 4)      3"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "insertcols!(x, 1, :B => [1,2,3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that `insertcols!` can be used to insert several columns to a data frame at once and that it performs broadcasting if needed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 1 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>a</th></tr><tr><th></th><th title=\"Int64\">Int64</th></tr></thead><tbody><tr><th>1</th><td>1</td></tr><tr><th>2</th><td>2</td></tr><tr><th>3</th><td>3</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& a\\\\\n",
       "\t\\hline\n",
       "\t& Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 \\\\\n",
       "\t2 & 2 \\\\\n",
       "\t3 & 3 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×1 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\n",
       "─────┼───────\n",
       "   1 │     1\n",
       "   2 │     2\n",
       "   3 │     3"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = DataFrame(a = [1, 2, 3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 4 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th><th>c</th><th>d</th></tr><tr><th></th><th title=\"Int64\">Int64</th><th title=\"String\">String</th><th title=\"Char\">Char</th><th title=\"Vector{Int64}\">Array…</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>x</td><td>a</td><td>[1, 2, 3]</td></tr><tr><th>2</th><td>2</td><td>x</td><td>b</td><td>[1, 2, 3]</td></tr><tr><th>3</th><td>3</td><td>x</td><td>c</td><td>[1, 2, 3]</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& a & b & c & d\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & String & Char & Array…\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & x & a & [1, 2, 3] \\\\\n",
       "\t2 & 2 & x & b & [1, 2, 3] \\\\\n",
       "\t3 & 3 & x & c & [1, 2, 3] \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a     \u001b[0m\u001b[1m b      \u001b[0m\u001b[1m c    \u001b[0m\u001b[1m d         \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m String \u001b[0m\u001b[90m Char \u001b[0m\u001b[90m Array…    \u001b[0m\n",
       "─────┼────────────────────────────────\n",
       "   1 │     1  x       a     [1, 2, 3]\n",
       "   2 │     2  x       b     [1, 2, 3]\n",
       "   3 │     3  x       c     [1, 2, 3]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "insertcols!(df, :b => \"x\", :c => 'a':'c', :d => Ref([1,2,3]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Interestingly we can emulate `hcat` mutating the data frame in-place using `insertcols!`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>2 rows × 1 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>a</th></tr><tr><th></th><th title=\"Int64\">Int64</th></tr></thead><tbody><tr><th>1</th><td>1</td></tr><tr><th>2</th><td>2</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& a\\\\\n",
       "\t\\hline\n",
       "\t& Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 \\\\\n",
       "\t2 & 2 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m2×1 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\n",
       "─────┼───────\n",
       "   1 │     1\n",
       "   2 │     2"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1 = DataFrame(a=[1,2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>2 rows × 2 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>b</th><th>c</th></tr><tr><th></th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th></tr></thead><tbody><tr><th>1</th><td>2</td><td>3</td></tr><tr><th>2</th><td>3</td><td>4</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& b & c\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 2 & 3 \\\\\n",
       "\t2 & 3 & 4 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m2×2 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m b     \u001b[0m\u001b[1m c     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼──────────────\n",
       "   1 │     2      3\n",
       "   2 │     3      4"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2 = DataFrame(b=[2,3], c=[3,4])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>2 rows × 3 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th><th>c</th></tr><tr><th></th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>2</td><td>3</td></tr><tr><th>2</th><td>2</td><td>3</td><td>4</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& a & b & c\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 2 & 3 \\\\\n",
       "\t2 & 2 & 3 & 4 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m2×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a     \u001b[0m\u001b[1m b     \u001b[0m\u001b[1m c     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼─────────────────────\n",
       "   1 │     1      2      3\n",
       "   2 │     2      3      4"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hcat(df1, df2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>2 rows × 1 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>a</th></tr><tr><th></th><th title=\"Int64\">Int64</th></tr></thead><tbody><tr><th>1</th><td>1</td></tr><tr><th>2</th><td>2</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& a\\\\\n",
       "\t\\hline\n",
       "\t& Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 \\\\\n",
       "\t2 & 2 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m2×1 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\n",
       "─────┼───────\n",
       "   1 │     1\n",
       "   2 │     2"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1 # df1 is not touched"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>2 rows × 3 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th><th>c</th></tr><tr><th></th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>2</td><td>3</td></tr><tr><th>2</th><td>2</td><td>3</td><td>4</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& a & b & c\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 2 & 3 \\\\\n",
       "\t2 & 2 & 3 & 4 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m2×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a     \u001b[0m\u001b[1m b     \u001b[0m\u001b[1m c     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼─────────────────────\n",
       "   1 │     1      2      3\n",
       "   2 │     2      3      4"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "insertcols!(df1, pairs(eachcol(df2))...)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>2 rows × 3 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th><th>c</th></tr><tr><th></th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>2</td><td>3</td></tr><tr><th>2</th><td>2</td><td>3</td><td>4</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& a & b & c\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 2 & 3 \\\\\n",
       "\t2 & 2 & 3 & 4 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m2×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a     \u001b[0m\u001b[1m b     \u001b[0m\u001b[1m c     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼─────────────────────\n",
       "   1 │     1      2      3\n",
       "   2 │     2      3      4"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1 # now we have changed df1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Subsetting/removing columns\n",
    "\n",
    "Let's create a new `DataFrame` `x` and show a few ways to create DataFrames with a subset of `x`'s columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 5 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>x5</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& x1 & x2 & x3 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)  (1, 3)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 2)  (2, 3)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 2)  (3, 3)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame([(i,j) for i in 1:3, j in 1:5], :auto)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we could do this by index:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 4 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x4</th><th>x5</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1 & x2 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 2) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 2) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 2)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 2)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[:, [1,2,4,5]] # use ! instead of : for non-copying operation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or by column name:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 2 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x4</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 1)</td><td>(1, 4)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 4)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 4)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x1 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 4) \\\\\n",
       "\t2 & (2, 1) & (2, 4) \\\\\n",
       "\t3 & (3, 1) & (3, 4) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×2 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x4     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────\n",
       "   1 │ (1, 1)  (1, 4)\n",
       "   2 │ (2, 1)  (2, 4)\n",
       "   3 │ (3, 1)  (3, 4)"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[:, [:x1, :x4]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also choose to keep or exclude columns by `Bool` (we need a vector whose length is the number of columns in the original `DataFrame`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 3 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x3</th><th>x5</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 1)</td><td>(1, 3)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 3)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 3)</td><td>(3, 5)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& x1 & x3 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 3) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 3) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 3) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────\n",
       "   1 │ (1, 1)  (1, 3)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 3)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 3)  (3, 5)"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[:, [true, false, true, false, true]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we create a single column `DataFrame`,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 1 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 1)</td></tr><tr><th>2</th><td>(2, 1)</td></tr><tr><th>3</th><td>(3, 1)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& x1\\\\\n",
       "\t\\hline\n",
       "\t& Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) \\\\\n",
       "\t2 & (2, 1) \\\\\n",
       "\t3 & (3, 1) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×1 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────\n",
       "   1 │ (1, 1)\n",
       "   2 │ (2, 1)\n",
       "   3 │ (3, 1)"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[:, [:x1]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and here we access the vector contained in column `:x1`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element Vector{Tuple{Int64, Int64}}:\n",
       " (1, 1)\n",
       " (2, 1)\n",
       " (3, 1)"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[!, :x1] # use : instead of ! to copy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element Vector{Tuple{Int64, Int64}}:\n",
       " (1, 1)\n",
       " (2, 1)\n",
       " (3, 1)"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.x1 # the same"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We could grab the same vector by column number"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element Vector{Tuple{Int64, Int64}}:\n",
       " (1, 1)\n",
       " (2, 1)\n",
       " (3, 1)"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[!, 1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that getting a single column returns it without copying while creating a new `DataFrame` performs a copy of the column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "false"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[!, 1] === x[!, [1]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "you can also use `Regex`, `All`, `Between` and `Not` from InvertedIndies.jl for column selection:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 2 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x1 & x2\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) \\\\\n",
       "\t2 & (2, 1) & (2, 2) \\\\\n",
       "\t3 & (3, 1) & (3, 2) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×2 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────\n",
       "   1 │ (1, 1)  (1, 2)\n",
       "   2 │ (2, 1)  (2, 2)\n",
       "   3 │ (3, 1)  (3, 2)"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[!, r\"[12]\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 4 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x2</th><th>x3</th><th>x4</th><th>x5</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x2 & x3 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 2) & (1, 3) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 2) & (2, 3) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 2) & (3, 3) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────\n",
       "   1 │ (1, 2)  (1, 3)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 2)  (2, 3)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 2)  (3, 3)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[!, Not(1)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 3 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x2</th><th>x3</th><th>x4</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td></tr><tr><th>2</th><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td></tr><tr><th>3</th><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 2) & (1, 3) & (1, 4) \\\\\n",
       "\t2 & (2, 2) & (2, 3) & (2, 4) \\\\\n",
       "\t3 & (3, 2) & (3, 3) & (3, 4) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────\n",
       "   1 │ (1, 2)  (1, 3)  (1, 4)\n",
       "   2 │ (2, 2)  (2, 3)  (2, 4)\n",
       "   3 │ (3, 2)  (3, 3)  (3, 4)"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[!, Between(:x2, :x4)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 4 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x3</th><th>x4</th><th>x5</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 1)</td><td>(1, 3)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 3)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 3)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1 & x3 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 3) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 3) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 3) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 3)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 3)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 3)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[!, Cols(:x1, Between(:x3, :x5))]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 4 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x3</th><th>x4</th><th>x5</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 1)</td><td>(1, 3)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 3)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 3)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1 & x3 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 3) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 3) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 3) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 3)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 3)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 3)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "select(x, :x1, Between(:x3, :x5), copycols=false) # the same as above"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "you can use `select` and `select!` functions to select a subset of columns from a data frame. `select` creates a new data frame and `select!` operates in place"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 5 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>x5</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& x1 & x2 & x3 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)  (1, 3)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 2)  (2, 3)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 2)  (3, 3)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = copy(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 2 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x1 & x2\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) \\\\\n",
       "\t2 & (2, 1) & (2, 2) \\\\\n",
       "\t3 & (3, 1) & (3, 2) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×2 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────\n",
       "   1 │ (1, 1)  (1, 2)\n",
       "   2 │ (2, 1)  (2, 2)\n",
       "   3 │ (3, 1)  (3, 2)"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2 = select(df, [1, 2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 3 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x3</th><th>x4</th><th>x5</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 3)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 3)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 3)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& x3 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 3) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 3) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 3) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────\n",
       "   1 │ (1, 3)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 3)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 3)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "select(df, Not([1, 2]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "by default `select` copies columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "false"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2[!, 1] === df[!, 1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "this can be avoided by using `copycols=false` keyword argument"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 2 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x1 & x2\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) \\\\\n",
       "\t2 & (2, 1) & (2, 2) \\\\\n",
       "\t3 & (3, 1) & (3, 2) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×2 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────\n",
       "   1 │ (1, 1)  (1, 2)\n",
       "   2 │ (2, 1)  (2, 2)\n",
       "   3 │ (3, 1)  (3, 2)"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2 = select(df, [1, 2], copycols=false)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "true"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2[!, 1] === df[!, 1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "using `select!` will modify the source data frame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 2 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x1 & x2\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) \\\\\n",
       "\t2 & (2, 1) & (2, 2) \\\\\n",
       "\t3 & (3, 1) & (3, 2) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×2 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────\n",
       "   1 │ (1, 1)  (1, 2)\n",
       "   2 │ (2, 1)  (2, 2)\n",
       "   3 │ (3, 1)  (3, 2)"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "select!(df, [1,2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "true"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df == df2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we create a copy of `x` and delete the 3rd column from the copy with `select!` and `Not`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 4 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x4</th><th>x5</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1 & x2 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 2) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 2) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 2)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 2)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z = copy(x)\n",
    "select!(z, Not(3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "alternatively we can achieve the same by using the `select` function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 4 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x4</th><th>x5</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1 & x2 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 2) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 2) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 2)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 2)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "select(x, Not(3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`x` stays unchanged"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 5 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>x5</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& x1 & x2 & x3 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)  (1, 3)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 2)  (2, 3)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 2)  (3, 3)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Views"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note, that you can also create a view of a `DataFrame` when we want a subset of its columns:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  1.020 μs (18 allocations: 1.64 KiB)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 3 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x3</th><th>x5</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 1)</td><td>(1, 3)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 3)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 3)</td><td>(3, 5)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& x1 & x3 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 3) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 3) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 3) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────\n",
       "   1 │ (1, 1)  (1, 3)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 3)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 3)  (3, 5)"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "@btime x[:, [1,3,5]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  214.464 ns (3 allocations: 256 bytes)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 3 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x3</th><th>x5</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 1)</td><td>(1, 3)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 3)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 3)</td><td>(3, 5)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& x1 & x3 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 3) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 3) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 3) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×3 SubDataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────\n",
       "   1 │ (1, 1)  (1, 3)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 3)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 3)  (3, 5)"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "@btime @view x[:, [1,3,5]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(now creation of the `view` is slow, but in the coming releases of the DataFrames.jl package it will become significantly faster)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Modify column by name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 5 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>x5</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& x1 & x2 & x3 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)  (1, 3)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 2)  (2, 3)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 2)  (3, 3)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame([(i,j) for i in 1:3, j in 1:5], :auto)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the following syntax, the existing column is modified without performing any copying (this is discouraged as it creates column alias)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 5 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>x5</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 2)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 2)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 2)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& x1 & x2 & x3 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 2) & (1, 2) & (1, 3) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 2) & (2, 2) & (2, 3) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 2) & (3, 2) & (3, 3) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────────────\n",
       "   1 │ (1, 2)  (1, 2)  (1, 3)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 2)  (2, 2)  (2, 3)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 2)  (3, 2)  (3, 3)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[!, :x1] = x[!, :x2]\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "this syntax is safer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element Vector{Tuple{Int64, Int64}}:\n",
       " (1, 2)\n",
       " (2, 2)\n",
       " (3, 2)"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[!, :x1] = x[:, :x2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also use the following syntax to add a new column at the end of a `DataFrame`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 6 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>x5</th><th>A</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Int64\">Int64</th></tr></thead><tbody><tr><th>1</th><td>(1, 2)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>(1, 5)</td><td>1</td></tr><tr><th>2</th><td>(2, 2)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>(2, 5)</td><td>2</td></tr><tr><th>3</th><td>(3, 2)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>(3, 5)</td><td>3</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccccc}\n",
       "\t& x1 & x2 & x3 & x4 & x5 & A\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple… & Tuple… & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 2) & (1, 2) & (1, 3) & (1, 4) & (1, 5) & 1 \\\\\n",
       "\t2 & (2, 2) & (2, 2) & (2, 3) & (2, 4) & (2, 5) & 2 \\\\\n",
       "\t3 & (3, 2) & (3, 2) & (3, 3) & (3, 4) & (3, 5) & 3 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×6 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\u001b[1m A     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼───────────────────────────────────────────────\n",
       "   1 │ (1, 2)  (1, 2)  (1, 3)  (1, 4)  (1, 5)      1\n",
       "   2 │ (2, 2)  (2, 2)  (2, 3)  (2, 4)  (2, 5)      2\n",
       "   3 │ (3, 2)  (3, 2)  (3, 3)  (3, 4)  (3, 5)      3"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[!, :A] = [1,2,3]\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A new column name will be added to our `DataFrame` with the following syntax as well:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 7 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>x5</th><th>A</th><th>B</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th></tr></thead><tbody><tr><th>1</th><td>(1, 2)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>(1, 5)</td><td>1</td><td>11</td></tr><tr><th>2</th><td>(2, 2)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>(2, 5)</td><td>2</td><td>12</td></tr><tr><th>3</th><td>(3, 2)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>(3, 5)</td><td>3</td><td>13</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccccc}\n",
       "\t& x1 & x2 & x3 & x4 & x5 & A & B\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple… & Tuple… & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 2) & (1, 2) & (1, 3) & (1, 4) & (1, 5) & 1 & 11 \\\\\n",
       "\t2 & (2, 2) & (2, 2) & (2, 3) & (2, 4) & (2, 5) & 2 & 12 \\\\\n",
       "\t3 & (3, 2) & (3, 2) & (3, 3) & (3, 4) & (3, 5) & 3 & 13 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×7 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\u001b[1m A     \u001b[0m\u001b[1m B     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼──────────────────────────────────────────────────────\n",
       "   1 │ (1, 2)  (1, 2)  (1, 3)  (1, 4)  (1, 5)      1     11\n",
       "   2 │ (2, 2)  (2, 2)  (2, 3)  (2, 4)  (2, 5)      2     12\n",
       "   3 │ (3, 2)  (3, 2)  (3, 3)  (3, 4)  (3, 5)      3     13"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.B = 11:13\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Find column name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 5 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>x5</th></tr><tr><th></th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th><th title=\"Tuple{Int64, Int64}\">Tuple…</th></tr></thead><tbody><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& x1 & x2 & x3 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)  (1, 3)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 2)  (2, 3)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 2)  (3, 3)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame([(i,j) for i in 1:3, j in 1:5], :auto)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can check if a column with a given name exists via"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "true"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hasproperty(x, :x1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and determine its index via"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "columnindex(x, :x2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Advanced ways of column selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "these are most useful for non-standard column names (e.g. containing spaces)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 2 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>column 2</th></tr><tr><th></th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>4</td></tr><tr><th>2</th><td>2</td><td>5</td></tr><tr><th>3</th><td>3</td><td>6</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x1 & column 2\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 4 \\\\\n",
       "\t2 & 2 & 5 \\\\\n",
       "\t3 & 3 & 6 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×2 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1    \u001b[0m\u001b[1m column 2 \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64    \u001b[0m\n",
       "─────┼─────────────────\n",
       "   1 │     1         4\n",
       "   2 │     2         5\n",
       "   3 │     3         6"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = DataFrame()\n",
    "df.x1 = 1:3\n",
    "df[!, \"column 2\"] = 4:6\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element Vector{Int64}:\n",
       " 4\n",
       " 5\n",
       " 6"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.\"column 2\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element Vector{Int64}:\n",
       " 4\n",
       " 5\n",
       " 6"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[:, \"column 2\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or you can interpolate column name using `:()` syntax"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x1\n",
      "[1, 2, 3]\n",
      "\n",
      "column 2\n",
      "[4, 5, 6]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for n in names(df)\n",
    "    println(n, \"\\n\", df.:($n), \"\\n\")\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Working on a collection of columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When using `eachcol` of a data frame the resulting object retains reference to its parent and e.g. can be queried with `getproperty`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 4 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th></tr><tr><th></th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>4</td><td>7</td><td>10</td></tr><tr><th>2</th><td>2</td><td>5</td><td>8</td><td>11</td></tr><tr><th>3</th><td>3</td><td>6</td><td>9</td><td>12</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1 & x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 4 & 7 & 10 \\\\\n",
       "\t2 & 2 & 5 & 8 & 11 \\\\\n",
       "\t3 & 3 & 6 & 9 & 12 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1    \u001b[0m\u001b[1m x2    \u001b[0m\u001b[1m x3    \u001b[0m\u001b[1m x4    \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼────────────────────────────\n",
       "   1 │     1      4      7     10\n",
       "   2 │     2      5      8     11\n",
       "   3 │     3      6      9     12"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = DataFrame(reshape(1:12, 3, 4), :auto)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<p>3×4 DataFrameColumns</p><div class=\"data-frame\"><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th></tr><tr><th></th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>4</td><td>7</td><td>10</td></tr><tr><th>2</th><td>2</td><td>5</td><td>8</td><td>11</td></tr><tr><th>3</th><td>3</td><td>6</td><td>9</td><td>12</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1 & x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 4 & 7 & 10 \\\\\n",
       "\t2 & 2 & 5 & 8 & 11 \\\\\n",
       "\t3 & 3 & 6 & 9 & 12 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrameColumns\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1    \u001b[0m\u001b[1m x2    \u001b[0m\u001b[1m x3    \u001b[0m\u001b[1m x4    \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼────────────────────────────\n",
       "   1 │     1      4      7     10\n",
       "   2 │     2      5      8     11\n",
       "   3 │     3      6      9     12"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ec_df = eachcol(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element Vector{Int64}:\n",
       " 1\n",
       " 2\n",
       " 3"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ec_df[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element Vector{Int64}:\n",
       " 1\n",
       " 2\n",
       " 3"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ec_df.x1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Transforming columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will get to this subject later in 10_transforms.ipynb notebook, but here let us just note that `select`, `select!`, `transform`, `transform!` and `combine` functions allow to generate new columns based on the old columns of a data frame.\n",
    "\n",
    "The general rules are the following:\n",
    "* `select` and `transform` always return the number of rows equal to the source data frame, while `combine` returns any number of rows (`combine` is allowed to *combine* rows of the source data frame)\n",
    "* `transform` retains columns from the old data frame\n",
    "* `select!` and `transform!` are in-place versions of `select` and `transform`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 4 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th></tr><tr><th></th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>4</td><td>7</td><td>10</td></tr><tr><th>2</th><td>2</td><td>5</td><td>8</td><td>11</td></tr><tr><th>3</th><td>3</td><td>6</td><td>9</td><td>12</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1 & x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 4 & 7 & 10 \\\\\n",
       "\t2 & 2 & 5 & 8 & 11 \\\\\n",
       "\t3 & 3 & 6 & 9 & 12 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1    \u001b[0m\u001b[1m x2    \u001b[0m\u001b[1m x3    \u001b[0m\u001b[1m x4    \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼────────────────────────────\n",
       "   1 │     1      4      7     10\n",
       "   2 │     2      5      8     11\n",
       "   3 │     3      6      9     12"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = DataFrame(reshape(1:12, 3, 4), :auto)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we add a new column `:res` that is a sum of columns `:x1` and `:x2`. A general syntax of transformations of this kind is:\n",
    "\n",
    "```\n",
    "source_columns => function_to_apply => target_column_name\n",
    "```\n",
    "then `function_to_apply` gets columns selected by `source_columns` as positional arguments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 5 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>res</th></tr><tr><th></th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>4</td><td>7</td><td>10</td><td>5</td></tr><tr><th>2</th><td>2</td><td>5</td><td>8</td><td>11</td><td>7</td></tr><tr><th>3</th><td>3</td><td>6</td><td>9</td><td>12</td><td>9</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& x1 & x2 & x3 & x4 & res\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 4 & 7 & 10 & 5 \\\\\n",
       "\t2 & 2 & 5 & 8 & 11 & 7 \\\\\n",
       "\t3 & 3 & 6 & 9 & 12 & 9 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1    \u001b[0m\u001b[1m x2    \u001b[0m\u001b[1m x3    \u001b[0m\u001b[1m x4    \u001b[0m\u001b[1m res   \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼───────────────────────────────────\n",
       "   1 │     1      4      7     10      5\n",
       "   2 │     2      5      8     11      7\n",
       "   3 │     3      6      9     12      9"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transform(df, [:x1, :x2] => (+) => :res)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One can omit passing `target_column_name` in which case it is automatically generated:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [],
   "source": [
    "using Statistics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>1 rows × 1 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1_x2_cor</th></tr><tr><th></th><th title=\"Float64\">Float64</th></tr></thead><tbody><tr><th>1</th><td>1.0</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& x1\\_x2\\_cor\\\\\n",
       "\t\\hline\n",
       "\t& Float64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1.0 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m1×1 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1_x2_cor \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Float64   \u001b[0m\n",
       "─────┼───────────\n",
       "   1 │       1.0"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "combine(df, [:x1, :x2] => cor)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that `combine` allowed the number of columns in the resulting data frame to be changed. If we used `select` instead it would automatically broadcast the return value to match the number of rouws of the source:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 1 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1_x2_cor</th></tr><tr><th></th><th title=\"Float64\">Float64</th></tr></thead><tbody><tr><th>1</th><td>1.0</td></tr><tr><th>2</th><td>1.0</td></tr><tr><th>3</th><td>1.0</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& x1\\_x2\\_cor\\\\\n",
       "\t\\hline\n",
       "\t& Float64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1.0 \\\\\n",
       "\t2 & 1.0 \\\\\n",
       "\t3 & 1.0 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×1 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1_x2_cor \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Float64   \u001b[0m\n",
       "─────┼───────────\n",
       "   1 │       1.0\n",
       "   2 │       1.0\n",
       "   3 │       1.0"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "select(df, [:x1, :x2] => cor)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to apply some function on each row of the source wrap it in `ByRow`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 3 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x1_x2_string</th></tr><tr><th></th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th><th title=\"String\">String</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>4</td><td>14</td></tr><tr><th>2</th><td>2</td><td>5</td><td>25</td></tr><tr><th>3</th><td>3</td><td>6</td><td>36</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& x1 & x2 & x1\\_x2\\_string\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & String\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 4 & 14 \\\\\n",
       "\t2 & 2 & 5 & 25 \\\\\n",
       "\t3 & 3 & 6 & 36 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1    \u001b[0m\u001b[1m x2    \u001b[0m\u001b[1m x1_x2_string \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m String       \u001b[0m\n",
       "─────┼────────────────────────────\n",
       "   1 │     1      4  14\n",
       "   2 │     2      5  25\n",
       "   3 │     3      6  36"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "select(df, :x1, :x2, [:x1, :x2] => ByRow(string))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Also if you want columns to be passed as a `NamedTuple` to a funcion (instead of being positional arguments) wrap them in `AsTable`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 3 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x1_x2_function</th></tr><tr><th></th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>4</td><td>5</td></tr><tr><th>2</th><td>2</td><td>5</td><td>7</td></tr><tr><th>3</th><td>3</td><td>6</td><td>9</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& x1 & x2 & x1\\_x2\\_function\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 4 & 5 \\\\\n",
       "\t2 & 2 & 5 & 7 \\\\\n",
       "\t3 & 3 & 6 & 9 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1    \u001b[0m\u001b[1m x2    \u001b[0m\u001b[1m x1_x2_function \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64          \u001b[0m\n",
       "─────┼──────────────────────────────\n",
       "   1 │     1      4               5\n",
       "   2 │     2      5               7\n",
       "   3 │     3      6               9"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "select(df, :x1, :x2, AsTable([:x1, :x2]) => x -> x.x1 + x.x2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For simplicity (as this functionality is often needed) there is a special treatement of `nrow` function that can be just passed as a transformation (without specifying of column selector):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 2 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>number_of_rows</th></tr><tr><th></th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>3</td></tr><tr><th>2</th><td>2</td><td>3</td></tr><tr><th>3</th><td>3</td><td>3</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x1 & number\\_of\\_rows\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 3 \\\\\n",
       "\t2 & 2 & 3 \\\\\n",
       "\t3 & 3 & 3 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×2 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1    \u001b[0m\u001b[1m number_of_rows \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64          \u001b[0m\n",
       "─────┼───────────────────────\n",
       "   1 │     1               3\n",
       "   2 │     2               3\n",
       "   3 │     3               3"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "select(df, :x1, nrow => \"number_of_rows\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(note that in `select` the number of rows is automatically broadcasted to match the number of rows of the source data frame)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally you can conveninently create multiple columns with one function, e.g.:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 3 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x1²</th><th>x1³</th></tr><tr><th></th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>1</td><td>1</td></tr><tr><th>2</th><td>2</td><td>4</td><td>8</td></tr><tr><th>3</th><td>3</td><td>9</td><td>27</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& x1 & x1² & x1³\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 1 & 1 \\\\\n",
       "\t2 & 2 & 4 & 8 \\\\\n",
       "\t3 & 3 & 9 & 27 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1    \u001b[0m\u001b[1m x1²   \u001b[0m\u001b[1m x1³   \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼─────────────────────\n",
       "   1 │     1      1      1\n",
       "   2 │     2      4      8\n",
       "   3 │     3      9     27"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "select(df, :x1, :x1 => ByRow(x -> [x ^ 2, x ^ 3]) => [\"x1²\", \"x1³\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or e.g. (this produces the same result)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>3 rows × 3 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x1²</th><th>x1³</th></tr><tr><th></th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th><th title=\"Int64\">Int64</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>1</td><td>1</td></tr><tr><th>2</th><td>2</td><td>4</td><td>8</td></tr><tr><th>3</th><td>3</td><td>9</td><td>27</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& x1 & x1² & x1³\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 1 & 1 \\\\\n",
       "\t2 & 2 & 4 & 8 \\\\\n",
       "\t3 & 3 & 9 & 27 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1    \u001b[0m\u001b[1m x1²   \u001b[0m\u001b[1m x1³   \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼─────────────────────\n",
       "   1 │     1      1      1\n",
       "   2 │     2      4      8\n",
       "   3 │     3      9     27"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "select(df, :x1, :x1 => (x -> DataFrame(\"x1²\" => x .^ 2, \"x1³\" => x .^ 3)) => AsTable)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that since DataFrames.jl row aggregation for wide tables is efficient. Here is an example of a wide table with `sum` (other standard reductions are similarly supported):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>1,000 rows × 10,000 columns (omitted printing of 9992 columns)</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>x5</th><th>x6</th><th>x7</th><th>x8</th></tr><tr><th></th><th title=\"Float64\">Float64</th><th title=\"Float64\">Float64</th><th title=\"Float64\">Float64</th><th title=\"Float64\">Float64</th><th title=\"Float64\">Float64</th><th title=\"Float64\">Float64</th><th title=\"Float64\">Float64</th><th title=\"Float64\">Float64</th></tr></thead><tbody><tr><th>1</th><td>0.0149088</td><td>0.177963</td><td>0.374975</td><td>0.694063</td><td>0.290585</td><td>0.296972</td><td>0.151378</td><td>0.0895086</td></tr><tr><th>2</th><td>0.520355</td><td>0.670122</td><td>0.387857</td><td>0.724383</td><td>0.0800228</td><td>0.248548</td><td>0.911363</td><td>0.36889</td></tr><tr><th>3</th><td>0.639562</td><td>0.0423618</td><td>0.779594</td><td>0.130453</td><td>0.173812</td><td>0.525416</td><td>0.324965</td><td>0.905649</td></tr><tr><th>4</th><td>0.839622</td><td>0.740699</td><td>0.360969</td><td>0.456934</td><td>0.398093</td><td>0.509447</td><td>0.335432</td><td>0.373952</td></tr><tr><th>5</th><td>0.967143</td><td>0.302672</td><td>0.706902</td><td>0.0653216</td><td>0.644417</td><td>0.976457</td><td>0.36895</td><td>0.80553</td></tr><tr><th>6</th><td>0.205168</td><td>0.683128</td><td>0.786909</td><td>0.819635</td><td>0.759663</td><td>0.588019</td><td>0.244011</td><td>0.888136</td></tr><tr><th>7</th><td>0.527184</td><td>0.22961</td><td>0.395737</td><td>0.712888</td><td>0.286281</td><td>0.0784137</td><td>0.214371</td><td>0.693681</td></tr><tr><th>8</th><td>0.951162</td><td>0.963781</td><td>0.176482</td><td>0.751149</td><td>0.394366</td><td>0.945845</td><td>0.62704</td><td>0.96774</td></tr><tr><th>9</th><td>0.536369</td><td>0.118179</td><td>0.958758</td><td>0.161565</td><td>0.722348</td><td>0.35534</td><td>0.734991</td><td>0.859331</td></tr><tr><th>10</th><td>0.711389</td><td>0.661071</td><td>0.450584</td><td>0.791275</td><td>0.0225357</td><td>0.582416</td><td>0.832604</td><td>0.0454298</td></tr><tr><th>11</th><td>0.103929</td><td>0.489357</td><td>0.943654</td><td>0.981498</td><td>0.906475</td><td>0.27605</td><td>0.319854</td><td>0.505438</td></tr><tr><th>12</th><td>0.806704</td><td>0.597994</td><td>0.293338</td><td>0.71464</td><td>0.821485</td><td>0.421753</td><td>0.578613</td><td>0.274216</td></tr><tr><th>13</th><td>0.870539</td><td>0.0866983</td><td>0.720978</td><td>0.957936</td><td>0.410978</td><td>0.132653</td><td>0.665372</td><td>0.593325</td></tr><tr><th>14</th><td>0.790207</td><td>0.957745</td><td>0.144518</td><td>0.988098</td><td>0.558975</td><td>0.855644</td><td>0.890067</td><td>0.0355403</td></tr><tr><th>15</th><td>0.800393</td><td>0.155563</td><td>0.21758</td><td>0.898822</td><td>0.751869</td><td>0.585648</td><td>0.993167</td><td>0.927108</td></tr><tr><th>16</th><td>0.898931</td><td>0.729694</td><td>0.279658</td><td>0.205926</td><td>0.596491</td><td>0.695755</td><td>0.695853</td><td>0.175926</td></tr><tr><th>17</th><td>0.939548</td><td>0.622013</td><td>0.32406</td><td>0.971788</td><td>0.778883</td><td>0.112</td><td>0.392893</td><td>0.968717</td></tr><tr><th>18</th><td>0.526344</td><td>0.615295</td><td>0.222202</td><td>0.302713</td><td>0.89399</td><td>0.0705911</td><td>0.0447971</td><td>0.529437</td></tr><tr><th>19</th><td>0.0779683</td><td>0.349777</td><td>0.168805</td><td>0.698644</td><td>0.131061</td><td>0.391454</td><td>0.147786</td><td>0.664002</td></tr><tr><th>20</th><td>0.966197</td><td>0.918847</td><td>0.445359</td><td>0.986339</td><td>0.364943</td><td>0.728386</td><td>0.850772</td><td>0.740342</td></tr><tr><th>21</th><td>0.666558</td><td>0.785516</td><td>0.65396</td><td>0.566016</td><td>0.790067</td><td>0.950925</td><td>0.759014</td><td>0.750961</td></tr><tr><th>22</th><td>0.230104</td><td>0.303686</td><td>0.396746</td><td>0.41203</td><td>0.836363</td><td>0.359704</td><td>0.573183</td><td>0.631113</td></tr><tr><th>23</th><td>0.0585193</td><td>0.982141</td><td>0.504897</td><td>0.0288279</td><td>0.99351</td><td>0.521489</td><td>0.782399</td><td>0.14326</td></tr><tr><th>24</th><td>0.298878</td><td>0.197828</td><td>0.921214</td><td>0.372391</td><td>0.531751</td><td>0.160731</td><td>0.310757</td><td>0.763507</td></tr><tr><th>25</th><td>0.380799</td><td>0.314726</td><td>0.574718</td><td>0.594419</td><td>0.436391</td><td>0.109001</td><td>0.824338</td><td>0.648367</td></tr><tr><th>26</th><td>0.0687591</td><td>0.930626</td><td>0.762947</td><td>0.339501</td><td>0.350716</td><td>0.641212</td><td>0.0405993</td><td>0.365176</td></tr><tr><th>27</th><td>0.861725</td><td>0.997935</td><td>0.00707363</td><td>0.702614</td><td>0.289486</td><td>0.699093</td><td>0.503938</td><td>0.53228</td></tr><tr><th>28</th><td>0.00543656</td><td>0.67684</td><td>0.979526</td><td>0.547557</td><td>0.797773</td><td>0.426561</td><td>0.766105</td><td>0.552505</td></tr><tr><th>29</th><td>0.698478</td><td>0.643354</td><td>0.545089</td><td>0.647449</td><td>0.62582</td><td>0.925163</td><td>0.0579145</td><td>0.241449</td></tr><tr><th>30</th><td>0.637793</td><td>0.112717</td><td>0.283882</td><td>0.359841</td><td>0.566783</td><td>0.447965</td><td>0.0586073</td><td>0.547599</td></tr><tr><th>&vellip;</th><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccccccc}\n",
       "\t& x1 & x2 & x3 & x4 & x5 & x6 & x7 & x8 & \\\\\n",
       "\t\\hline\n",
       "\t& Float64 & Float64 & Float64 & Float64 & Float64 & Float64 & Float64 & Float64 & \\\\\n",
       "\t\\hline\n",
       "\t1 & 0.0149088 & 0.177963 & 0.374975 & 0.694063 & 0.290585 & 0.296972 & 0.151378 & 0.0895086 & $\\dots$ \\\\\n",
       "\t2 & 0.520355 & 0.670122 & 0.387857 & 0.724383 & 0.0800228 & 0.248548 & 0.911363 & 0.36889 & $\\dots$ \\\\\n",
       "\t3 & 0.639562 & 0.0423618 & 0.779594 & 0.130453 & 0.173812 & 0.525416 & 0.324965 & 0.905649 & $\\dots$ \\\\\n",
       "\t4 & 0.839622 & 0.740699 & 0.360969 & 0.456934 & 0.398093 & 0.509447 & 0.335432 & 0.373952 & $\\dots$ \\\\\n",
       "\t5 & 0.967143 & 0.302672 & 0.706902 & 0.0653216 & 0.644417 & 0.976457 & 0.36895 & 0.80553 & $\\dots$ \\\\\n",
       "\t6 & 0.205168 & 0.683128 & 0.786909 & 0.819635 & 0.759663 & 0.588019 & 0.244011 & 0.888136 & $\\dots$ \\\\\n",
       "\t7 & 0.527184 & 0.22961 & 0.395737 & 0.712888 & 0.286281 & 0.0784137 & 0.214371 & 0.693681 & $\\dots$ \\\\\n",
       "\t8 & 0.951162 & 0.963781 & 0.176482 & 0.751149 & 0.394366 & 0.945845 & 0.62704 & 0.96774 & $\\dots$ \\\\\n",
       "\t9 & 0.536369 & 0.118179 & 0.958758 & 0.161565 & 0.722348 & 0.35534 & 0.734991 & 0.859331 & $\\dots$ \\\\\n",
       "\t10 & 0.711389 & 0.661071 & 0.450584 & 0.791275 & 0.0225357 & 0.582416 & 0.832604 & 0.0454298 & $\\dots$ \\\\\n",
       "\t11 & 0.103929 & 0.489357 & 0.943654 & 0.981498 & 0.906475 & 0.27605 & 0.319854 & 0.505438 & $\\dots$ \\\\\n",
       "\t12 & 0.806704 & 0.597994 & 0.293338 & 0.71464 & 0.821485 & 0.421753 & 0.578613 & 0.274216 & $\\dots$ \\\\\n",
       "\t13 & 0.870539 & 0.0866983 & 0.720978 & 0.957936 & 0.410978 & 0.132653 & 0.665372 & 0.593325 & $\\dots$ \\\\\n",
       "\t14 & 0.790207 & 0.957745 & 0.144518 & 0.988098 & 0.558975 & 0.855644 & 0.890067 & 0.0355403 & $\\dots$ \\\\\n",
       "\t15 & 0.800393 & 0.155563 & 0.21758 & 0.898822 & 0.751869 & 0.585648 & 0.993167 & 0.927108 & $\\dots$ \\\\\n",
       "\t16 & 0.898931 & 0.729694 & 0.279658 & 0.205926 & 0.596491 & 0.695755 & 0.695853 & 0.175926 & $\\dots$ \\\\\n",
       "\t17 & 0.939548 & 0.622013 & 0.32406 & 0.971788 & 0.778883 & 0.112 & 0.392893 & 0.968717 & $\\dots$ \\\\\n",
       "\t18 & 0.526344 & 0.615295 & 0.222202 & 0.302713 & 0.89399 & 0.0705911 & 0.0447971 & 0.529437 & $\\dots$ \\\\\n",
       "\t19 & 0.0779683 & 0.349777 & 0.168805 & 0.698644 & 0.131061 & 0.391454 & 0.147786 & 0.664002 & $\\dots$ \\\\\n",
       "\t20 & 0.966197 & 0.918847 & 0.445359 & 0.986339 & 0.364943 & 0.728386 & 0.850772 & 0.740342 & $\\dots$ \\\\\n",
       "\t21 & 0.666558 & 0.785516 & 0.65396 & 0.566016 & 0.790067 & 0.950925 & 0.759014 & 0.750961 & $\\dots$ \\\\\n",
       "\t22 & 0.230104 & 0.303686 & 0.396746 & 0.41203 & 0.836363 & 0.359704 & 0.573183 & 0.631113 & $\\dots$ \\\\\n",
       "\t23 & 0.0585193 & 0.982141 & 0.504897 & 0.0288279 & 0.99351 & 0.521489 & 0.782399 & 0.14326 & $\\dots$ \\\\\n",
       "\t24 & 0.298878 & 0.197828 & 0.921214 & 0.372391 & 0.531751 & 0.160731 & 0.310757 & 0.763507 & $\\dots$ \\\\\n",
       "\t25 & 0.380799 & 0.314726 & 0.574718 & 0.594419 & 0.436391 & 0.109001 & 0.824338 & 0.648367 & $\\dots$ \\\\\n",
       "\t26 & 0.0687591 & 0.930626 & 0.762947 & 0.339501 & 0.350716 & 0.641212 & 0.0405993 & 0.365176 & $\\dots$ \\\\\n",
       "\t27 & 0.861725 & 0.997935 & 0.00707363 & 0.702614 & 0.289486 & 0.699093 & 0.503938 & 0.53228 & $\\dots$ \\\\\n",
       "\t28 & 0.00543656 & 0.67684 & 0.979526 & 0.547557 & 0.797773 & 0.426561 & 0.766105 & 0.552505 & $\\dots$ \\\\\n",
       "\t29 & 0.698478 & 0.643354 & 0.545089 & 0.647449 & 0.62582 & 0.925163 & 0.0579145 & 0.241449 & $\\dots$ \\\\\n",
       "\t30 & 0.637793 & 0.112717 & 0.283882 & 0.359841 & 0.566783 & 0.447965 & 0.0586073 & 0.547599 & $\\dots$ \\\\\n",
       "\t$\\dots$ & $\\dots$ & $\\dots$ & $\\dots$ & $\\dots$ & $\\dots$ & $\\dots$ & $\\dots$ & $\\dots$ &  \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m1000×10000 DataFrame\u001b[0m\n",
       "\u001b[1m  Row \u001b[0m│\u001b[1m x1        \u001b[0m\u001b[1m x2        \u001b[0m\u001b[1m x3        \u001b[0m\u001b[1m x4        \u001b[0m\u001b[1m x5        \u001b[0m\u001b[1m x6        \u001b[0m\u001b[1m x7  \u001b[0m ⋯\n",
       "\u001b[1m      \u001b[0m│\u001b[90m Float64   \u001b[0m\u001b[90m Float64   \u001b[0m\u001b[90m Float64   \u001b[0m\u001b[90m Float64   \u001b[0m\u001b[90m Float64   \u001b[0m\u001b[90m Float64   \u001b[0m\u001b[90m Floa\u001b[0m ⋯\n",
       "──────┼─────────────────────────────────────────────────────────────────────────\n",
       "    1 │ 0.0149088  0.177963   0.374975   0.694063   0.290585   0.296972   0.15 ⋯\n",
       "    2 │ 0.520355   0.670122   0.387857   0.724383   0.0800228  0.248548   0.91\n",
       "    3 │ 0.639562   0.0423618  0.779594   0.130453   0.173812   0.525416   0.32\n",
       "    4 │ 0.839622   0.740699   0.360969   0.456934   0.398093   0.509447   0.33\n",
       "    5 │ 0.967143   0.302672   0.706902   0.0653216  0.644417   0.976457   0.36 ⋯\n",
       "    6 │ 0.205168   0.683128   0.786909   0.819635   0.759663   0.588019   0.24\n",
       "    7 │ 0.527184   0.22961    0.395737   0.712888   0.286281   0.0784137  0.21\n",
       "    8 │ 0.951162   0.963781   0.176482   0.751149   0.394366   0.945845   0.62\n",
       "    9 │ 0.536369   0.118179   0.958758   0.161565   0.722348   0.35534    0.73 ⋯\n",
       "   10 │ 0.711389   0.661071   0.450584   0.791275   0.0225357  0.582416   0.83\n",
       "   11 │ 0.103929   0.489357   0.943654   0.981498   0.906475   0.27605    0.31\n",
       "  ⋮   │     ⋮          ⋮          ⋮          ⋮          ⋮          ⋮           ⋱\n",
       "  991 │ 0.318699   0.855621   0.89789    0.186005   0.757165   0.537527   0.52\n",
       "  992 │ 0.385526   0.0811559  0.479614   0.163361   0.0102084  0.649546   0.65 ⋯\n",
       "  993 │ 0.678458   0.557108   0.202757   0.493311   0.343117   0.922908   0.26\n",
       "  994 │ 0.424295   0.78012    0.577681   0.274228   0.225563   0.29553    0.39\n",
       "  995 │ 0.481554   0.0441882  0.700181   0.317677   0.0846761  0.949135   0.45\n",
       "  996 │ 0.401528   0.658903   0.438309   0.211217   0.211515   0.441902   0.02 ⋯\n",
       "  997 │ 0.58985    0.909857   0.354735   0.442829   0.438322   0.847835   0.84\n",
       "  998 │ 0.871944   0.821183   0.725636   0.521645   0.230773   0.506747   0.47\n",
       "  999 │ 0.702069   0.399582   0.377331   0.542735   0.38631    0.555332   0.74\n",
       " 1000 │ 0.70703    0.166208   0.45823    0.0682235  0.458422   0.836939   0.58 ⋯\n",
       "\u001b[36m                                               9994 columns and 979 rows omitted\u001b[0m"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "large_df = DataFrame(rand(1000, 10000), :auto)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  0.694893 seconds (2.20 M allocations: 132.235 MiB, 3.58% gc time, 98.51% compilation time)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>1,000 rows × 1 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>sum</th></tr><tr><th></th><th title=\"Float64\">Float64</th></tr></thead><tbody><tr><th>1</th><td>5010.73</td></tr><tr><th>2</th><td>5055.37</td></tr><tr><th>3</th><td>4987.2</td></tr><tr><th>4</th><td>5016.2</td></tr><tr><th>5</th><td>4978.86</td></tr><tr><th>6</th><td>5018.0</td></tr><tr><th>7</th><td>5011.07</td></tr><tr><th>8</th><td>4991.09</td></tr><tr><th>9</th><td>4932.78</td></tr><tr><th>10</th><td>5017.35</td></tr><tr><th>11</th><td>4997.14</td></tr><tr><th>12</th><td>4994.29</td></tr><tr><th>13</th><td>5020.93</td></tr><tr><th>14</th><td>4917.99</td></tr><tr><th>15</th><td>5023.11</td></tr><tr><th>16</th><td>5007.8</td></tr><tr><th>17</th><td>4982.31</td></tr><tr><th>18</th><td>5010.66</td></tr><tr><th>19</th><td>4962.08</td></tr><tr><th>20</th><td>5013.45</td></tr><tr><th>21</th><td>5008.7</td></tr><tr><th>22</th><td>5050.03</td></tr><tr><th>23</th><td>5009.0</td></tr><tr><th>24</th><td>5016.84</td></tr><tr><th>25</th><td>5015.97</td></tr><tr><th>26</th><td>5009.97</td></tr><tr><th>27</th><td>4966.94</td></tr><tr><th>28</th><td>4961.31</td></tr><tr><th>29</th><td>4992.18</td></tr><tr><th>30</th><td>5037.17</td></tr><tr><th>&vellip;</th><td>&vellip;</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& sum\\\\\n",
       "\t\\hline\n",
       "\t& Float64\\\\\n",
       "\t\\hline\n",
       "\t1 & 5010.73 \\\\\n",
       "\t2 & 5055.37 \\\\\n",
       "\t3 & 4987.2 \\\\\n",
       "\t4 & 5016.2 \\\\\n",
       "\t5 & 4978.86 \\\\\n",
       "\t6 & 5018.0 \\\\\n",
       "\t7 & 5011.07 \\\\\n",
       "\t8 & 4991.09 \\\\\n",
       "\t9 & 4932.78 \\\\\n",
       "\t10 & 5017.35 \\\\\n",
       "\t11 & 4997.14 \\\\\n",
       "\t12 & 4994.29 \\\\\n",
       "\t13 & 5020.93 \\\\\n",
       "\t14 & 4917.99 \\\\\n",
       "\t15 & 5023.11 \\\\\n",
       "\t16 & 5007.8 \\\\\n",
       "\t17 & 4982.31 \\\\\n",
       "\t18 & 5010.66 \\\\\n",
       "\t19 & 4962.08 \\\\\n",
       "\t20 & 5013.45 \\\\\n",
       "\t21 & 5008.7 \\\\\n",
       "\t22 & 5050.03 \\\\\n",
       "\t23 & 5009.0 \\\\\n",
       "\t24 & 5016.84 \\\\\n",
       "\t25 & 5015.97 \\\\\n",
       "\t26 & 5009.97 \\\\\n",
       "\t27 & 4966.94 \\\\\n",
       "\t28 & 4961.31 \\\\\n",
       "\t29 & 4992.18 \\\\\n",
       "\t30 & 5037.17 \\\\\n",
       "\t$\\dots$ & $\\dots$ \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m1000×1 DataFrame\u001b[0m\n",
       "\u001b[1m  Row \u001b[0m│\u001b[1m sum     \u001b[0m\n",
       "\u001b[1m      \u001b[0m│\u001b[90m Float64 \u001b[0m\n",
       "──────┼─────────\n",
       "    1 │ 5010.73\n",
       "    2 │ 5055.37\n",
       "    3 │ 4987.2\n",
       "    4 │ 5016.2\n",
       "    5 │ 4978.86\n",
       "    6 │ 5018.0\n",
       "    7 │ 5011.07\n",
       "    8 │ 4991.09\n",
       "    9 │ 4932.78\n",
       "   10 │ 5017.35\n",
       "   11 │ 4997.14\n",
       "  ⋮   │    ⋮\n",
       "  991 │ 4997.01\n",
       "  992 │ 5003.09\n",
       "  993 │ 4984.85\n",
       "  994 │ 4933.86\n",
       "  995 │ 4952.49\n",
       "  996 │ 5015.75\n",
       "  997 │ 5013.19\n",
       "  998 │ 4976.72\n",
       "  999 │ 5015.42\n",
       " 1000 │ 5054.93\n",
       "\u001b[36m979 rows omitted\u001b[0m"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "@time select(large_df, AsTable(:) => ByRow(sum) => :sum)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  0.010603 seconds (19.67 k allocations: 1.073 MiB)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div class=\"data-frame\"><p>1,000 rows × 1 columns</p><table class=\"data-frame\"><thead><tr><th></th><th>sum</th></tr><tr><th></th><th title=\"Float64\">Float64</th></tr></thead><tbody><tr><th>1</th><td>5010.73</td></tr><tr><th>2</th><td>5055.37</td></tr><tr><th>3</th><td>4987.2</td></tr><tr><th>4</th><td>5016.2</td></tr><tr><th>5</th><td>4978.86</td></tr><tr><th>6</th><td>5018.0</td></tr><tr><th>7</th><td>5011.07</td></tr><tr><th>8</th><td>4991.09</td></tr><tr><th>9</th><td>4932.78</td></tr><tr><th>10</th><td>5017.35</td></tr><tr><th>11</th><td>4997.14</td></tr><tr><th>12</th><td>4994.29</td></tr><tr><th>13</th><td>5020.93</td></tr><tr><th>14</th><td>4917.99</td></tr><tr><th>15</th><td>5023.11</td></tr><tr><th>16</th><td>5007.8</td></tr><tr><th>17</th><td>4982.31</td></tr><tr><th>18</th><td>5010.66</td></tr><tr><th>19</th><td>4962.08</td></tr><tr><th>20</th><td>5013.45</td></tr><tr><th>21</th><td>5008.7</td></tr><tr><th>22</th><td>5050.03</td></tr><tr><th>23</th><td>5009.0</td></tr><tr><th>24</th><td>5016.84</td></tr><tr><th>25</th><td>5015.97</td></tr><tr><th>26</th><td>5009.97</td></tr><tr><th>27</th><td>4966.94</td></tr><tr><th>28</th><td>4961.31</td></tr><tr><th>29</th><td>4992.18</td></tr><tr><th>30</th><td>5037.17</td></tr><tr><th>&vellip;</th><td>&vellip;</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& sum\\\\\n",
       "\t\\hline\n",
       "\t& Float64\\\\\n",
       "\t\\hline\n",
       "\t1 & 5010.73 \\\\\n",
       "\t2 & 5055.37 \\\\\n",
       "\t3 & 4987.2 \\\\\n",
       "\t4 & 5016.2 \\\\\n",
       "\t5 & 4978.86 \\\\\n",
       "\t6 & 5018.0 \\\\\n",
       "\t7 & 5011.07 \\\\\n",
       "\t8 & 4991.09 \\\\\n",
       "\t9 & 4932.78 \\\\\n",
       "\t10 & 5017.35 \\\\\n",
       "\t11 & 4997.14 \\\\\n",
       "\t12 & 4994.29 \\\\\n",
       "\t13 & 5020.93 \\\\\n",
       "\t14 & 4917.99 \\\\\n",
       "\t15 & 5023.11 \\\\\n",
       "\t16 & 5007.8 \\\\\n",
       "\t17 & 4982.31 \\\\\n",
       "\t18 & 5010.66 \\\\\n",
       "\t19 & 4962.08 \\\\\n",
       "\t20 & 5013.45 \\\\\n",
       "\t21 & 5008.7 \\\\\n",
       "\t22 & 5050.03 \\\\\n",
       "\t23 & 5009.0 \\\\\n",
       "\t24 & 5016.84 \\\\\n",
       "\t25 & 5015.97 \\\\\n",
       "\t26 & 5009.97 \\\\\n",
       "\t27 & 4966.94 \\\\\n",
       "\t28 & 4961.31 \\\\\n",
       "\t29 & 4992.18 \\\\\n",
       "\t30 & 5037.17 \\\\\n",
       "\t$\\dots$ & $\\dots$ \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m1000×1 DataFrame\u001b[0m\n",
       "\u001b[1m  Row \u001b[0m│\u001b[1m sum     \u001b[0m\n",
       "\u001b[1m      \u001b[0m│\u001b[90m Float64 \u001b[0m\n",
       "──────┼─────────\n",
       "    1 │ 5010.73\n",
       "    2 │ 5055.37\n",
       "    3 │ 4987.2\n",
       "    4 │ 5016.2\n",
       "    5 │ 4978.86\n",
       "    6 │ 5018.0\n",
       "    7 │ 5011.07\n",
       "    8 │ 4991.09\n",
       "    9 │ 4932.78\n",
       "   10 │ 5017.35\n",
       "   11 │ 4997.14\n",
       "  ⋮   │    ⋮\n",
       "  991 │ 4997.01\n",
       "  992 │ 5003.09\n",
       "  993 │ 4984.85\n",
       "  994 │ 4933.86\n",
       "  995 │ 4952.49\n",
       "  996 │ 5015.75\n",
       "  997 │ 5013.19\n",
       "  998 │ 4976.72\n",
       "  999 │ 5015.42\n",
       " 1000 │ 5054.93\n",
       "\u001b[36m979 rows omitted\u001b[0m"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "@time select(large_df, AsTable(:) => ByRow(sum) => :sum)"
   ]
  }
 ],
 "metadata": {
  "@webio": {
   "lastCommId": null,
   "lastKernelId": null
  },
  "kernelspec": {
   "display_name": "Julia 1.7.0",
   "language": "julia",
   "name": "julia-1.7"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}