Ch3/03_Bag_of_N_Grams.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Bag of N-Grams.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.8"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "V_hosDAcZwSq"
      },
      "source": [
        "## Bag of N-Grams\n",
        "\n",
        "##### One hot encoding, BoW and TF-IDF treat words as independent units. There is no notion of phrases or word ordering. Bag of Ngrams (BoN) approach tries to remedy this. It does so by breaking text into chunks of n countigous words/tokens. This can help us capture some context, which earlier approaches could not do. Let us see how it works using the same toy corpus we used in earlier examples. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "wE4F2qCx7NvL",
        "outputId": "1021a10b-9239-4cab-afe5-c08f8910bcef",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "#our corpus\n",
        "documents = [\"Dog bites man.\", \"Man bites dog.\", \"Dog eats meat.\", \"Man eats food.\"]\n",
        "\n",
        "processed_docs = [doc.lower().replace(\".\",\"\") for doc in documents]\n",
        "processed_docs"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['dog bites man', 'man bites dog', 'dog eats meat', 'man eats food']"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 1
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "khNajktSVfnS",
        "colab_type": "text"
      },
      "source": [
        "##### CountVectorizer, which we used for BoW, can be used for getting a Bag of N-grams representation as well, using its ngram_range argument. The code snippet below shows how: "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "j7-VReVY5jZr",
        "outputId": "d4dfff0a-b6e4-4488-f6c7-525722e2fb6d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 108
        }
      },
      "source": [
        "from sklearn.feature_extraction.text import CountVectorizer\n",
        "\n",
        "\n",
        "#Ngram vectorization example with count vectorizer and uni, bi, trigrams\n",
        "count_vect = CountVectorizer(ngram_range=(1,3))\n",
        "\n",
        "#Build a BOW representation for the corpus\n",
        "bow_rep = count_vect.fit_transform(processed_docs)\n",
        "\n",
        "#Look at the vocabulary mapping\n",
        "print(\"Our vocabulary: \", count_vect.vocabulary_)\n",
        "\n",
        "#see the BOW rep for first 2 documents\n",
        "print(\"BoW representation for 'dog bites man': \", bow_rep[0].toarray())\n",
        "print(\"BoW representation for 'man bites dog: \",bow_rep[1].toarray())\n",
        "\n",
        "#Get the representation using this vocabulary, for a new text\n",
        "temp = count_vect.transform([\"dog and dog are friends\"])\n",
        "\n",
        "print(\"Bow representation for 'dog and dog are friends':\", temp.toarray())"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Our vocabulary:  {'dog': 3, 'bites': 0, 'man': 12, 'dog bites': 4, 'bites man': 2, 'dog bites man': 5, 'man bites': 13, 'bites dog': 1, 'man bites dog': 14, 'eats': 8, 'meat': 17, 'dog eats': 6, 'eats meat': 10, 'dog eats meat': 7, 'food': 11, 'man eats': 15, 'eats food': 9, 'man eats food': 16}\n",
            "BoW representation for 'dog bites man':  [[1 0 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0]]\n",
            "BoW representation for 'man bites dog:  [[1 1 0 1 0 0 0 0 0 0 0 0 1 1 1 0 0 0]]\n",
            "Bow representation for 'dog and dog are friends': [[0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZNG5mUeWVohV",
        "colab_type": "text"
      },
      "source": [
        "##### Note that the number of features (and hence the size of the feature vector) increased a lot for the same data, compared to the ther single word based representations!!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "Cy5f149j8nKh",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FSAoOQ2vVnGY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KUu_4EYnVfnk",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}