Week-05.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Files!\n",
    "\n",
    "This week we're learning the basics of handling files for reading in and writing out.  We're used to our operating systems handling all these things for us, so we need to reproduce those backend mechanisms ourselves.\n",
    "\n",
    "The basic pattern is:\n",
    "\n",
    "1. Open the file in whichever mode you need\n",
    "2. Do stuff to that file.\n",
    "3. Close the file.\n",
    "\n",
    "There are ways of handling files in sort of a one shot, but we'll get to that later.\n",
    "\n",
    "## The cursor\n",
    "\n",
    "Don't curse the cursor!  When you open a file for reading, there's an invisible cursor that the system places within that file.  It starts at the very beginning of the file and moves forward at your direction.  This means you can direct it to go forward character by character or line by line.  But you can never move backwards!  This means that you cannot 'reread' a file once you've already done a read action to it.  Once the cursor has found the end of the file, it stops and waits.\n",
    "\n",
    "For example:\n",
    "\n",
    "1. You open a file for reading\n",
    "2. You use `.read()` on that file.  The cursor is now at the end of the file.\n",
    "3. You cannot then use `.readlines()` on it after because `.readlines()` is going to attempt to start reading from the end of that file.  You'll get nothing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hello, I would like to science.\n",
      "Please show me where the science is.\n",
      "I am a meat popcicle.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "my_file = open('smalltext.txt', 'rt')\n",
    "\n",
    "all_the_text = my_file.read()\n",
    "\n",
    "print(all_the_text) # just the first 10 lines, please\n",
    "\n",
    "maybemore = my_file.read()\n",
    "\n",
    "print(maybemore)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Printing these out sort of hide what's happening in these strings. So let's use some handy interactive stuff."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Hello, I would like to science.\\nPlease show me where the science is.\\nI am a meat popcicle.'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_the_text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "''"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "maybemore"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So `maybemore` worked, but is empty.  The cursor had no more text to go through, so it just gave us an empty string.  But let's look at this a different way and see if we can halt the cursor in the middle of a file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A told B, and B told C, \"I'll meet you at the top of the coconut tree.\"\n",
      "\n",
      "\"Wheee!\" said D to E F G, \"I'll beat you to the top of the coconut tree.\"\n",
      "\n",
      "Chicka chicka boom boom! Will there be enough room? Here comes H up the coconut tree,\n",
      "\n",
      "and I and J and tag-along K, all on their way up the coconut tree.\n",
      "\n",
      "Chicka chicka boom boom! Will there be enough room? Look who's coming! L M N O P!\n",
      "\n"
     ]
    }
   ],
   "source": [
    "my_file = open('boomboom.txt', 'rt')\n",
    "\n",
    "for justdothis5times in range(5):\n",
    "    print(my_file.readline())\n",
    "    \n",
    "therest = my_file.read()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "And Q R S! And T U V! Still more - W! And X Y Z!\n",
      "The whole alphabet up the - Oh, no! Chicka chicka... BOOM! BOOM!\n",
      "Skit skat skoodle doot. Flip flop flee. Everybody running to the coconut tree.\n",
      "Mamas and papas and uncles and aunts hug their little dears, then dust their pants.\n",
      "\"Help us up,\" cried A B C.\n",
      "Next from the pileup skinned-knee D and stubbed-toe E and patched-up F. Then comes G all out of breath.\n",
      "H is tangled up with I. J and K are about to cry. L is knotted like a tie.\n",
      "M is looped. N is stopped. O is twisted alley-oop. Skit skat skoodle doot. Flip flop flee.\n",
      "Look who's coming! It's black-eyed P, Q R S, and loose-tooth T. Then U V W wiggle-jiggle free.\n",
      "Last to come X Y Z. And the sun goes down on the coconut tree...\n",
      "But - chicka chicka boom boom! Look, there's a full moon.\n",
      "A is out of bed, and this is what he said, \"Dare double dare, you can't catch me.\n",
      "Chicka chicka BOOM! BOOM!Chicka chicka BOOM! BOOM!\n",
      "I'll beat you to the top of the coconut tree.\"\n",
      "Chicka chicka BOOM! BOOM!\n"
     ]
    }
   ],
   "source": [
    "print(therest)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What happened here?  We can see that the `.readline()` bit grabbed the first 5 lines and then the `.read()` got the rest of the lines.  There also seems to be extra newlines happening in the first section?\n",
    "\n",
    "I used a for loop with `range(5)` to repeat `.readline()` 5 times.  This meant it acted 5 independent times, so it printed out 5 lines that each ended in `\\n`, which means that gets rendered as an extra newline, because there's no text right after it.\n",
    "\n",
    "At this point, the cursor is sitting at the beginning of line 6 just waiting.  When I call `.read()` it goes through the remaining portion of the file.  There's no extra newlines happening because there's text to sit in those slots."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A problem to work through\n",
    "\n",
    "Here's something silly, but it's a task.\n",
    "\n",
    "Read through a text file and change it so that the lines alternate between upper and lower case (starting with upper), then write out the new file. \n",
    "\n",
    "So we want something like this:\n",
    "```\n",
    "LINE 1\n",
    "line 2\n",
    "LINE 3\n",
    "line 4\n",
    "```\n",
    "\n",
    "Let's break this down:\n",
    "\n",
    "1. Read in the file\n",
    "2. Transform the text\n",
    "3. Write out the file\n",
    "\n",
    "We can go ahead and set up items 1 and 3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "file_in = open('boomboom.txt', 'rt')\n",
    "\n",
    "write_me = file_in.read()\n",
    "\n",
    "file_out = open('newboom.txt', 'wt')\n",
    "\n",
    "file_out.write(write_me)\n",
    "\n",
    "file_in.close()\n",
    "file_out.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the code in the cell just above, we can see that we aren't transforming the text yet, but we are at leat set up for the read in and write out.  While we're playing with things, let's comment out the write stuff and just use print statements."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A told B, and B told C, \"I'll meet you at the top of the coconut tree.\"\n",
      "\"Wheee!\" said D to E F G, \"I'll beat you to the top of the coconut tree.\"\n",
      "Chicka chicka boom boom! Will there be enough room? Here comes H up the coconut tree,\n",
      "and I and J and tag-along K, all on their way up the coconut tree.\n",
      "Chicka chicka boom boom! Will there be enough room? Look who's coming! L M N O P!\n",
      "And Q R S! And T U V! Still more - W! And X Y Z!\n",
      "The whole alphabet up the - Oh, no! Chicka chicka... BOOM! BOOM!\n",
      "Skit skat skoodle doot. Flip flop flee. Everybody running to the coconut tree.\n",
      "Mamas and papas and uncles and aunts hug their little dears, then dust their pants.\n",
      "\"Help us up,\" cried A B C.\n",
      "Next from the pileup skinned-knee D and stubbed-toe E and patched-up F. Then comes G all out of breath.\n",
      "H is tangled up with I. J and K are about to cry. L is knotted like a tie.\n",
      "M is looped. N is stopped. O is twisted alley-oop. Skit skat skoodle doot. Flip flop flee.\n",
      "Look who's coming! It's black-eyed P, Q R S, and loose-tooth T. Then U V W wiggle-jiggle free.\n",
      "Last to come X Y Z. And the sun goes down on the coconut tree...\n",
      "But - chicka chicka boom boom! Look, there's a full moon.\n",
      "A is out of bed, and this is what he said, \"Dare double dare, you can't catch me.\n",
      "Chicka chicka BOOM! BOOM!Chicka chicka BOOM! BOOM!\n",
      "I'll beat you to the top of the coconut tree.\"\n",
      "Chicka chicka BOOM! BOOM!\n"
     ]
    }
   ],
   "source": [
    "file_in = open('boomboom.txt', 'rt')\n",
    "\n",
    "write_me = file_in.read()\n",
    "\n",
    "\n",
    "print(write_me)\n",
    "# file_out = open('newboom.txt', 'wt')\n",
    "\n",
    "# file_out.write(write_me)\n",
    "\n",
    "file_in.close()\n",
    "# file_out.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great! Now let's think of an approach for how to do this.  We know that we can use `.upper()` and `.lower()` to transform the text, but the issue is: how do we alternate lines of text?\n",
    "\n",
    "Think of it this way: we need to move through the file in terms of pairs.  One upper, one lower then one upper and one lower.\n",
    "\n",
    "We can use the cursor to our advantage here.  If we're already iterating through reading a file using a for loop, e.g.\n",
    "\n",
    "```python\n",
    "for line in file_in:\n",
    "    print(line)\n",
    "\n",
    "```\n",
    "\n",
    "We're just using this for loop as a convenient way to move the cursor one by one.  So can we move thorugh two lines within that for loop?  We can't just use line twice becuase it'll always be the second line, but we have another method in our pockets:  `.readline()`.  As we saw with our `range(5)` example, we can call `.readline()` an arbitrary number of times.  It won't mess up our for loop, but it will give us the next line, and push the cursor forward by one.\n",
    "\n",
    "In other words:\n",
    "\n",
    "\n",
    "```python\n",
    "\n",
    "# let's say the cursor starts at 0\n",
    "\n",
    "for line in file_in: # cursor + 1; cursor is now 1\n",
    "    print(line)      # cursor doesn't change; cursor remains 1\n",
    "    print(file_in.readline()) # cursor + 1; cursor is now 2\n",
    "    \n",
    "# now this will run again\n",
    "\n",
    "# ...\n",
    "\n",
    "# cursor is at 2 from the previous loop \n",
    "\n",
    "for line in file_in: # cursor + 1; cursor is now 3\n",
    "    print(line)      # cursor doesn't change; cursor remains 3\n",
    "    print(file_in.readline()) # cursor + 1; cursor is now 4\n",
    "    \n",
    "# and so on\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Let's play with this using just a regular counter and an accumulator pattern.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1, 1, 1, 1, 1, 1, 1, 1, 1, 1]\n",
      "2\n",
      "4\n",
      "6\n",
      "8\n",
      "10\n",
      "12\n",
      "14\n",
      "16\n",
      "18\n",
      "20\n"
     ]
    }
   ],
   "source": [
    "total = 0\n",
    "\n",
    "lotsofones = [1] * 10\n",
    "print(lotsofones)\n",
    "\n",
    "for one in lotsofones:\n",
    "    total = total + one\n",
    "    total = total + 1\n",
    "    print(total)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "So plunking all of this in together:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A TOLD B, AND B TOLD C, \"I'LL MEET YOU AT THE TOP OF THE COCONUT TREE.\"\n",
      "\"wheee!\" said d to e f g, \"i'll beat you to the top of the coconut tree.\"\n",
      "CHICKA CHICKA BOOM BOOM! WILL THERE BE ENOUGH ROOM? HERE COMES H UP THE COCONUT TREE,\n",
      "and i and j and tag-along k, all on their way up the coconut tree.\n",
      "CHICKA CHICKA BOOM BOOM! WILL THERE BE ENOUGH ROOM? LOOK WHO'S COMING! L M N O P!\n",
      "and q r s! and t u v! still more - w! and x y z!\n",
      "THE WHOLE ALPHABET UP THE - OH, NO! CHICKA CHICKA... BOOM! BOOM!\n",
      "skit skat skoodle doot. flip flop flee. everybody running to the coconut tree.\n",
      "MAMAS AND PAPAS AND UNCLES AND AUNTS HUG THEIR LITTLE DEARS, THEN DUST THEIR PANTS.\n",
      "\"help us up,\" cried a b c.\n",
      "NEXT FROM THE PILEUP SKINNED-KNEE D AND STUBBED-TOE E AND PATCHED-UP F. THEN COMES G ALL OUT OF BREATH.\n",
      "h is tangled up with i. j and k are about to cry. l is knotted like a tie.\n",
      "M IS LOOPED. N IS STOPPED. O IS TWISTED ALLEY-OOP. SKIT SKAT SKOODLE DOOT. FLIP FLOP FLEE.\n",
      "look who's coming! it's black-eyed p, q r s, and loose-tooth t. then u v w wiggle-jiggle free.\n",
      "LAST TO COME X Y Z. AND THE SUN GOES DOWN ON THE COCONUT TREE...\n",
      "but - chicka chicka boom boom! look, there's a full moon.\n",
      "A IS OUT OF BED, AND THIS IS WHAT HE SAID, \"DARE DOUBLE DARE, YOU CAN'T CATCH ME.\n",
      "chicka chicka boom! boom!chicka chicka boom! boom!\n",
      "I'LL BEAT YOU TO THE TOP OF THE COCONUT TREE.\"\n",
      "chicka chicka boom! boom!\n"
     ]
    }
   ],
   "source": [
    "file_in = open('boomboom.txt', 'rt')\n",
    "\n",
    "new_text = []\n",
    "\n",
    "for line in file_in:\n",
    "    new_text.append(line.upper())\n",
    "    new_text.append(file_in.readline().lower())\n",
    "\n",
    "write_me = \"\".join(new_text) # I retained the newlines as they came in from the original file\n",
    "                             # so I don't need to put them back in\n",
    "\n",
    "print(write_me)\n",
    "# file_out = open('newboom.txt', 'wt')\n",
    "\n",
    "# file_out.write(write_me)\n",
    "\n",
    "# file_in.close()\n",
    "# file_out.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'A TOLD B, AND B TOLD C, \"I\\'LL MEET YOU AT THE TOP OF THE COCONUT TREE.\"\\n\"wheee!\" said d to e f g, \"i\\'ll beat you to the top of the coconut tree.\"\\nCHICKA CHICKA BOOM BOOM! WILL THERE BE ENOUGH ROOM? HERE COMES H UP THE COCONUT TREE,\\nand i and j and tag-along k, all on their way up the coconut tree.\\nCHICKA CHICKA BOOM BOOM! WILL THERE BE ENOUGH ROOM? LOOK WHO\\'S COMING! L M N O P!\\nand q r s! and t u v! still more - w! and x y z!\\nTHE WHOLE ALPHABET UP THE - OH, NO! CHICKA CHICKA... BOOM! BOOM!\\nskit skat skoodle doot. flip flop flee. everybody running to the coconut tree.\\nMAMAS AND PAPAS AND UNCLES AND AUNTS HUG THEIR LITTLE DEARS, THEN DUST THEIR PANTS.\\n\"help us up,\" cried a b c.\\nNEXT FROM THE PILEUP SKINNED-KNEE D AND STUBBED-TOE E AND PATCHED-UP F. THEN COMES G ALL OUT OF BREATH.\\nh is tangled up with i. j and k are about to cry. l is knotted like a tie.\\nM IS LOOPED. N IS STOPPED. O IS TWISTED ALLEY-OOP. SKIT SKAT SKOODLE DOOT. FLIP FLOP FLEE.\\nlook who\\'s coming! it\\'s black-eyed p, q r s, and loose-tooth t. then u v w wiggle-jiggle free.\\nLAST TO COME X Y Z. AND THE SUN GOES DOWN ON THE COCONUT TREE...\\nbut - chicka chicka boom boom! look, there\\'s a full moon.\\nA IS OUT OF BED, AND THIS IS WHAT HE SAID, \"DARE DOUBLE DARE, YOU CAN\\'T CATCH ME.\\nchicka chicka boom! boom!chicka chicka boom! boom!\\nI\\'LL BEAT YOU TO THE TOP OF THE COCONUT TREE.\"\\nchicka chicka boom! boom!'"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}