Recurrent_neural_networks.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Recurrent neural networks\n",
    "\n",
    "Example script to generate text from Nietzsche's writings.\n",
    "At least 20 epochs are required before the generated text\n",
    "starts sounding coherent.\n",
    "\n",
    "It is recommended to run this script on GPU, as recurrent\n",
    "networks are quite computationally intensive.\n",
    "If you try this script on new data, make sure your corpus\n",
    "has at least ~100k characters. ~1M is better."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Activation\n",
    "from keras.layers import LSTM\n",
    "from keras.optimizers import RMSprop\n",
    "from keras.utils.data_utils import get_file\n",
    "import numpy as np\n",
    "import random\n",
    "import sys"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "corpus length: 600893\n",
      "total chars: 57\n",
      "nb sequences: 200285\n",
      "Vectorization...\n",
      "Build model...\n",
      "\n",
      "--------------------------------------------------\n",
      "Iteration 1\n",
      "Epoch 1/1\n",
      "200285/200285 [==============================] - 229s - loss: 1.9891   \n",
      "\n",
      "----- diversity: 0.2\n",
      "----- Generating with seed: \"hich bravely admits this to itself, very\"\n",
      "hich bravely admits this to itself, very man as the present to the self the sount the sensition to the self the pressing the sense to the soun the sensition of the soun the sount the sensition of the present and strengs of a something the sensition of the something the something the presing and consideral and so the presing and such a something the self the present to the soun the man and there one man and sperman the man the world the \n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"hich bravely admits this to itself, very\"\n",
      "hich bravely admits this to itself, very one serions henese on the comparity might be there the possing and acting one man in man the semparions and speraping and emporate and intince to the man the suncer and sperrariessialy the self with one as the somether to higher sensition and sempan to statt and ow is only the man the present and same an indeed and comparity the stinces of it is in has strengity: there to the perpoped on the free\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"hich bravely admits this to itself, very\"\n",
      "hich bravely admits this to itself, very and other only becomporer strencre, a time becifl, cering upod in therebelfulwity.\n",
      "y\n",
      "emans bo with someatity at beingruations\n",
      "the sens as the point as everymanther who leeg, redied\n",
      "becomithated mare,\n",
      "of\n",
      "theinlisrifity the gernat of the one, and our with we being sura\n",
      "verenies, suthat and soed beloon to sens-\"long in\n",
      "asing of that the cpsinst\n",
      "sosili nichation--a burits of wish manin\n",
      "great\n",
      "as and n\n",
      "----- diversity: 1.2\n",
      "----- Generating with seed: \"hich bravely admits this to itself, very\"\n",
      "hich bravely admits this to itself, very\n",
      "beond\n",
      "a\n",
      "counter daces, it we )o\n",
      "post this morely, dids, such a utculogiting herscem,\n",
      "he reevingss lusts\n",
      "bto lave, he\n",
      "its\n",
      "immy myslat\n",
      "lyasuut\n",
      "wan a terstantibtientate as a liferely\n",
      "beaory was which ghem!\n",
      "apoun, not reverowon not\n",
      "cersuop phifoon that, ther man, bo in means tear of a  sores thac.\n",
      "chiut man he\n",
      ". their one esple; by e rafir\n",
      "just\n",
      "chysp-imaginity\n",
      "or\n",
      "alpoven.\n",
      " ther oad, years--very phess\n",
      "--------------------------------------------------\n",
      "Iteration 2\n",
      "Epoch 1/1\n",
      "200285/200285 [==============================] - 220s - loss: 1.6334   \n",
      "\n",
      "----- diversity: 0.2\n",
      "----- Generating with seed: \"th\n",
      "regard to fichte. it is a question wh\"\n",
      "th\n",
      "regard to fichte. it is a question which the more the work and as a more and and the concertion of conclusion of the more a more and a precisely the concertion of the such a more and the such a prociest the self-does the such a such a the more and the procient of the sense of the more experience of the such a more and and the sensition of the such a more conterpouration of the self-desint and a the sense and more conscious and and a \n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"th\n",
      "regard to fichte. it is a question wh\"\n",
      "th\n",
      "regard to fichte. it is a question which\n",
      "knows of conterpouration part a proves the greater concertion of the himself--and not such a life a conscience of succifical will a great such a more pater of him. in the more concertion and a comparity of more and a present in the german the philosopher as the experiun which the sught and an extertation of the works which a develop the the act a conscious and above and and a plained and every\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"th\n",
      "regard to fichte. it is a question wh\"\n",
      "th\n",
      "regard to fichte. it is a question which,\n",
      "not a ftrinciey, at onyly. whichs, hitherty athemps or\n",
      "aytingticison of\n",
      "thus been weces: the seever, morl, sting be had philosopher\n",
      "the lacking which mexcurish onegetiseowing a udablever\" a froms largually\n",
      "somean no of how above nouwh so necessarior kind and him--the other amfull sufficrent to not what how meant, at protesticarraycent and artor ancient is the great the act perhaps a\n",
      "superfici\n",
      "----- diversity: 1.2\n",
      "----- Generating with seed: \"th\n",
      "regard to fichte. it is a question wh\"\n",
      "th\n",
      "regard to fichte. it is a question what only at cule a striiced. wichin upmorthen, coven was i whowals as simely ourge, \"bur lied the coryer? not everything onkeriming thed comnigion, inleith.\n",
      "\n",
      "unouthry, uncentestetscuately, not exprest pleventess such way\n",
      "which thinking\n",
      "sectud;, a maten-without haghouratly \"prodiinoser of alressed petsence of the work the\n",
      "freed (in raght abvesorion out at thone wangn,\n",
      "asorde\n",
      "natestine. pariricding e\n",
      "--------------------------------------------------\n",
      "Iteration 3\n",
      "Epoch 1/1\n",
      "200285/200285 [==============================] - 206s - loss: 1.5439   \n",
      "\n",
      "----- diversity: 0.2\n",
      "----- Generating with seed: \" wait upon\n",
      "consciousness with the reques\"\n",
      " wait upon\n",
      "consciousness with the request and the self the conscience and conscience of the sensation of the sensible to the stricted the sense of the conscience of the sensuality of the conscience and the sense of the such a probounder to the words to the probably to the strength of the words of the sense of the common the sensible such sense to the sense of the conscience of the precisely the sense of the sense of the striction of the\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \" wait upon\n",
      "consciousness with the reques\"\n",
      " wait upon\n",
      "consciousness with the request science to be fundian called the first call of pre: it is decever the contrary is for the noolzaged are of the conceination of the soul and that its only for the will of the worst of his his noological sensation of the higher, and the science and most and to the power of prevense always in the concesse of the whole, and necessary as a gains, it is a conscience, there is the regarded the superfin\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \" wait upon\n",
      "consciousness with the reques\"\n",
      " wait upon\n",
      "consciousness with the request are it\n",
      "is effect, thereby, in itself\n",
      "are it, all organss, and the\n",
      "eablise noo\n",
      "thought, the right is a musicidy had beingly as the mainific dreaderal consciently a slude to fies of the oldaages its discifer misserbable every ventivation to sself eully\n",
      "out of hercivations of world we\n",
      "sanstack it in evers. have are is god much a correarfly, as well to\n",
      "compuls of thougn\n",
      "errore that an an, that remas\n",
      "----- diversity: 1.2\n",
      "----- Generating with seed: \" wait upon\n",
      "consciousness with the reques\"\n",
      " wait upon\n",
      "consciousness with the requestne; they, op artible confould?--i whethers attaid, yet regard let eyss as in upon the distrustful,  happy may be evieical, courues, tis know suciifts,\n",
      "ill fear, head bunder, and andudo.\n",
      " i though goody, sasfinies: fraremebly work hid-\n",
      "=but finadience of new in\n",
      "idmetially\n",
      "is the cannibry philaposely ack to bioker?\n",
      "ything will oxiry for comely to it barer, are of is men. this vent manifiaking, rath\n",
      "--------------------------------------------------\n",
      "Iteration 4\n",
      "Epoch 1/1\n",
      "200285/200285 [==============================] - 208s - loss: 1.4997   \n",
      "\n",
      "----- diversity: 0.2\n",
      "----- Generating with seed: \"tely expedient, but upon the permanently\"\n",
      "tely expedient, but upon the permanently and the most themselves the great the consequence in the self-and the fact that is a man of the sense to the consequence of the most the all the stronger and the same as the most the light of the most always the moral and consequence of the same better the subjection of the considered that is the most always the subjection and suffering of the most the strengthess of the sentiment and the thing o\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"tely expedient, but upon the permanently\"\n",
      "tely expedient, but upon the permanently be instinct does all the procratic assugation of the soul of his formaling and simply demo-more who the power of woman is the problem of the refined himself, certain the transion and desire, and the same of chimpess of the subjection and complent of the same addigination that is a most respect which man of the philosophy in the entire interest of the glows of the tart man the truth as reason and \n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"tely expedient, but upon the permanently\"\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tely expedient, but upon the permanently him. therethe casual\n",
      "easidily knowing for much the crual formy with the spenitional intertious, as. has he compted where one through seinal himself taste, this conduces, powerful; when ye. he parifal,\n",
      "and\n",
      "extent a feel they generally astrocison and not assumisto--will, know under\n",
      "obliment of the laturs\n",
      "ancience,on cleasuse, posption, and which rage symple. the\n",
      "relarnt\n",
      "re kanf\n",
      "latament\n",
      "say ot\n",
      "-thi\n",
      "----- diversity: 1.2\n",
      "----- Generating with seed: \"tely expedient, but upon the permanently\"\n",
      "tely expedient, but upon the permanently naturily live\n",
      "motive\n",
      "tradless him--wh aresalod it daypt\n",
      "(not typise.b; when unveads to politiveline preduct \"fundernaity himself\n",
      "in dipbat, such oc -ligo:\n",
      "he destluwike propalical weaknessually one?\n",
      "it aspend\n",
      "have\n",
      "dread,\" no\n",
      "dembland when oherne himself and impulse willvou, which disite?\n",
      "\n",
      "them! he botwer-damucean logerid oud of it gecarning attlinual truthes, philosophersias, what only volwondest\n",
      "--------------------------------------------------\n",
      "Iteration 5\n",
      "Epoch 1/1\n",
      "200285/200285 [==============================] - 207s - loss: 1.4692   \n",
      "\n",
      "----- diversity: 0.2\n",
      "----- Generating with seed: \"al), to do things not in\n",
      "accordance with\"\n",
      "al), to do things not in\n",
      "accordance with the sense of the sentiment the sense of the sentiment to a such a problem of the same the sentiment the same beate the same the thing the sentiment the sense of the sense of the stands the order to the more the sentiment the sentiment the sentiment that the sense of the say to the that it is the strength of the more such a stands the strength of the that the sense of the sentiment of the sentimen\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"al), to do things not in\n",
      "accordance with\"\n",
      "al), to do things not in\n",
      "accordance with the conscience and results of the cause of the holds more in the soul of the way to the very the tase is new dending and relations, the more and nomminal not as the danger we can have the reason and an under that he that is no\n",
      "stranger, but under has and assertific ordening the more former without the conception of the prosence to the respect that the famely sentiment the sense of the existence t\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"al), to do things not in\n",
      "accordance with\"\n",
      "al), to do things not in\n",
      "accordance with the moral does around from or greatest anomy and it it only intentionity, has view of the mires;\n",
      "so to have are\n",
      "often\n",
      "his ta.\"ng this penting\n",
      "hachand before more and soul--not at the thing humanion--shatt and entire, that perhaps once it sympatumely all\n",
      "but too soul, a noble as grees she to, not\n",
      "in crevents, well-endude to that malls alsething new tand the  portly idea to an one is an a man and n\n",
      "----- diversity: 1.2\n",
      "----- Generating with seed: \"al), to do things not in\n",
      "accordance with\"\n",
      "al), to do things not in\n",
      "accordance within bewin?\" free, outhne? hacts, but it tought by book, and revelpent\n",
      "readly althainstitu, into generals upon horsmnts and surmonus, it would viroty, aleclated hative togething, he is i\n",
      "bedool. in the gonestbonce upset mind themselves came \"there\n",
      "is difficultion:n\n",
      "who knowm nows in\n",
      "themetion the ration in bearcers that or\n",
      "every onds meching out diedortfrmine upotress.\n",
      "\"hologn, nature wellest mmelf \n",
      "--------------------------------------------------\n",
      "Iteration 6\n",
      "Epoch 1/1\n",
      "200285/200285 [==============================] - 195s - loss: 1.4472   \n",
      "\n",
      "----- diversity: 0.2\n",
      "----- Generating with seed: \" conditions, and begins a new work with \"\n",
      " conditions, and begins a new work with an art of a man in the stands and any something of the strength, and even and in the consideration of mankind and the soul of the most something in the striving and interpreter and the most sense of mankind and the formerous and any such a presented to the stands of the formerous such a something of the belief in the senses of the soul of the stronger of the self-consideration of the protention of\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \" conditions, and begins a new work with \"\n",
      " conditions, and begins a new work with a interpretentimental the man of the formerous and superfrential contraps that the thing and takes so such and the pride and the self-protenty of badness, in the are and the will into the stands and worker of the self said to such as the most soul of the preuitation of hend a most appearance and trities, which may not any such an any castes and sense and the \"noting one proises and science and sen\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \" conditions, and begins a new work with \"\n",
      " conditions, and begins a new work with greatests us, and double of most divinificac, into\n",
      "perhaps--onouragionence, as occur lookschell; under us in the \"thations,\n",
      "spiritured to estally interest is upon therefore to and for iftiness\n",
      "up pleathen for much. he is a nouble and product is presenting as mytemgnely\n",
      "it was in. our myselfd; it it love like secrets to inl is the higher dles after weal morality and walth of other \"ghenered ea abou\n",
      "----- diversity: 1.2\n",
      "----- Generating with seed: \" conditions, and begins a new work with \"\n",
      " conditions, and begins a new work with sepre of\n",
      "onnged by\n",
      "esacmia. will sers them insensus belief in the\n",
      "unifention--of the most peamenting\n",
      "algably happiness,\n",
      "would eeterarisely mad\n",
      "hexen and\n",
      "renoty\n",
      "any mons came something prarying\n",
      "ordinary\n",
      "of the something togarlitures where can as a it is invention. icase in itself that dree existuieh commproural\n",
      "dasolt, and its\n",
      "disinable with we most is not of germaning inducable, many.\n",
      "\n",
      "179. i inte\n",
      "--------------------------------------------------\n",
      "Iteration 7\n",
      "Epoch 1/1\n",
      "200285/200285 [==============================] - 213s - loss: 1.4305   \n",
      "\n",
      "----- diversity: 0.2\n",
      "----- Generating with seed: \"\n",
      "assumption of plurality always presuppo\"\n",
      "\n",
      "assumption of plurality always presupposes and the man and struction of the problem, and which a problem, and which a contemplation of the superior the decover and expression of the strange the standard the propers of the problem, and and the strange of the strange of the strange of the strange of the proposite the struction of the problem, and the strange of the problem, as the strange of the subject, and in the standard the moral sta\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"\n",
      "assumption of plurality always presuppo\"\n",
      "\n",
      "assumption of plurality always presuppote the superior of the proporier at the sense itself have and in the semution of god and action the theor, when the any intellectual in the former of expressest of the profound in the still aother of the most in the good and necessarily the spirit and refined and any present in problem, the strength of god and religion the\n",
      "religion the the happiness of a still something\n",
      "man in a reternism, at all \n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"\n",
      "assumption of plurality always presuppo\"\n",
      "\n",
      "assumption of plurality always presuppoted real pessednences among tolk exertention, shil\n",
      "and be folpfeqque espicurianed by protiou\n",
      "will really found epectow--and pur his certain, and invert again contemparic\n",
      "\"spirit that is nother of this, not withare things whatever affoen and may not shadibur the is nowing what precreaturen effoction over escignined\" inned on \"y admire to the will give instinct there impell pleasure more thing, fone\n",
      "----- diversity: 1.2\n",
      "----- Generating with seed: \"\n",
      "assumption of plurality always presuppo\"\n",
      "\n",
      "assumption of plurality always presupposes.\n",
      "\n",
      "6 ench. in this il myself\n",
      "as impeopledned in voinger. soceentime men, ar, werthing, but inessilition\n",
      "yee, a loud light best such his, what werign educehold of men of great,al ordervation ol. to diyahive quadtious byly moral, away puttity espicioted\" everys saninadiving, gaerative hopef-mply seevs hit even it can that the expliitures, yea-s stickiwn\n",
      "and intinctive): such appiauly its didnishm\n",
      "--------------------------------------------------\n",
      "Iteration 8\n",
      "Epoch 1/1\n",
      "200285/200285 [==============================] - 210s - loss: 1.4169   \n",
      "\n",
      "----- diversity: 0.2\n",
      "----- Generating with seed: \"rowed from\n",
      "the past. the idea and the im\"\n",
      "rowed from\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the past. the idea and the immediately and the soul of the speaks are the good and the standard and the free of the prove also the seems to the problem of the prove and the subserved to the speaks of stands to a man importunated to the substance, and the speaks of the problem of the speaks of the substanted to the speaks of the substants of the sense and the more strongest the sense of the worst the profound of the speak of t\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"rowed from\n",
      "the past. the idea and the im\"\n",
      "rowed from\n",
      "the past. the idea and the importunated the same without the desired. he with the significantial, as the deep the strength of the deed as the stronger of which the rest and his distrustful it will not among the tray that is the same of his now belongs and the assemerned to the deal explanable like been constitution, to doubling the remain conquers of freedom and as the of the sense--a mean a much as the will to which has mean\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"rowed from\n",
      "the past. the idea and the im\"\n",
      "rowed from\n",
      "the past. the idea and the impo-moness a do the purites of the\n",
      "appear from lavents are namely continurnure at all prefering volners of his always sometimns about with the\n",
      "historids. within comminds to old his\n",
      "tachmies. breans of ourst life.-motive as procording errors and for puritality avbewlow in self-exclins\n",
      "himself\n",
      "that li moruwady she intellectual theness of by will himself. was constant enjoy as not to seet of bownred r\n",
      "----- diversity: 1.2\n",
      "----- Generating with seed: \"rowed from\n",
      "the past. the idea and the im\"\n",
      "rowed from\n",
      "the past. the idea and the impelse view.o\"; at aseral,\n",
      "nou do fam-assume wele dek-as joy absild of experiencen should not its, in a reselfuveity\n",
      "for thmels, but his\n",
      "otel value couster and more bowizisness of human real\n",
      "play his other;\n",
      "indescapic\n",
      "sculuma\n",
      "; and and no must a aflo all hand, should unmen mean a cosmedially adgreak almost excortanize morality. others,\n",
      "on as\"--navele, true to whatever has me.\n",
      "that really ongors awi\n",
      "--------------------------------------------------\n",
      "Iteration 9\n",
      "Epoch 1/1\n",
      "200285/200285 [==============================] - 196s - loss: 1.4055   \n",
      "\n",
      "----- diversity: 0.2\n",
      "----- Generating with seed: \"d propagating themselves--they will\n",
      "be t\"\n",
      "d propagating themselves--they will\n",
      "be the sensation of the and the consideration of the world and significan the stronger of the individual and something the fact that is the soul of the self-consideration of the same to seems to the something the self-man the fact that is the consideration of the soul of the self-man in the consideration of the soul of the standard and the standard to the serve of the soul of the spectance of the self\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"d propagating themselves--they will\n",
      "be t\"\n",
      "d propagating themselves--they will\n",
      "be the other from the self-man in the interpretation of the strange of the future. and that here the self-of the christice with is a feeling the weaker of himself, and the self-matters and every restractions, and the whole will and even the indignioration of the potting, for their avers and appears and all the strength of the self-fact that all serve of this actually does not to be a constance and str\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"d propagating themselves--they will\n",
      "be t\"\n",
      "d propagating themselves--they will\n",
      "be turnes is cest to alse and tastejent and it eraceo\"\"\"\n",
      "as served, there is the\n",
      "remisicianlymine, a side\n",
      "of tereing in him. and one are uncertain, and thingsh recomes in time\n",
      "right the laasing, but he find even in men bestable, and tor necessary doate to is--ourselves lovety, on an all them\n",
      "the among\n",
      "i thereby had clases and arttilatism, and intelladiofited fitt--time of vittadiss\n",
      "an are have been th\n",
      "----- diversity: 1.2\n",
      "----- Generating with seed: \"d propagating themselves--they will\n",
      "be t\"\n",
      "d propagating themselves--they will\n",
      "be too\n",
      "lackings traycained to may natural necessar--springsismd. here aboth\n",
      "libker sa\n",
      "doubth\"men causies\n",
      "of the immediated, femptacenal to discrean\n",
      "itself; well deparaal but become. herself to what is sension more\n",
      "defeniuty. and contricating time on efferd to femin\" decaiteries and win though them. and may which\n",
      "do mons\n",
      "crood just be living spear in, thenoenies.\n",
      "\n",
      "peihmin, or, in him sensulation,\n",
      "inled\n",
      "--------------------------------------------------\n",
      "Iteration 10\n",
      "Epoch 1/1\n",
      "200285/200285 [==============================] - 193s - loss: 1.3964   \n",
      "\n",
      "----- diversity: 0.2\n",
      "----- Generating with seed: \"s.--but who\n",
      "could do me this service! an\"\n",
      "s.--but who\n",
      "could do me this service! and as a something the process of the conceal and the self-dender and any oftendation of the senses and the actual the morality and the present to the concealed and the will to the contempt of the most altore of the interpretations, and itself, and the senses and the person and the morality of the senses to see the think of the senses and the problem, and the sense of a sort of the senses to a soul \n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"s.--but who\n",
      "could do me this service! an\"\n",
      "s.--but who\n",
      "could do me this service! and to be science of the power of the senses and happenested the discovically the idea of the enough, and and the thinker of the contempt, and alto a suffer\n",
      "the that betterious every self the expedience and the connected it is the sufferther, as an experience; there perpetuation of the self-same the contempts and as an experience? on the old one is the philosophers of new the power and hence and the\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"s.--but who\n",
      "could do me this service! an\"\n",
      "s.--but who\n",
      "could do me this service! and persenest; the he feel or the\n",
      "more understoodment of do the rilly had of first sentially liflses to the view\n",
      "looking itself; the\n",
      "arrurous itself; hell. they weretoroor is a \"self designiest is littlewismity, which good! no\n",
      "has es it belief in fact, and that hitherto doubt has been claims--and could atteanged\n",
      "the could have may being from our own obtaines of the right, they are europe\n",
      "as the doxt\n",
      "----- diversity: 1.2\n",
      "----- Generating with seed: \"s.--but who\n",
      "could do me this service! an\"\n",
      "s.--but who\n",
      "could do me this service! and would and perpltiating most full usferedy,\" mistake urvers, the \"not, what we aid innou\n",
      "thom, furnitime, as too appear, proied, out of\n",
      "the touatity is emblobroguans, is ascreitscrearsubl\n",
      "certain ersout that us ok the honourly and fee, but it of naturalthnal dnes--usive according in our first-sound\n",
      "of\n",
      "it\n",
      "itpell confineds which being from peoplatisisihousnce, the lifeiled\n",
      "byt\"gorasis man\" cannot l\n",
      "--------------------------------------------------\n",
      "Iteration 11\n",
      "Epoch 1/1\n",
      "200285/200285 [==============================] - 196s - loss: 1.3879   \n",
      "\n",
      "----- diversity: 0.2\n",
      "----- Generating with seed: \"sychological observation.=--that reflect\"\n",
      "sychological observation.=--that reflect of the soul of the sentiment and the contrarience to the strength of the sentiment and subjection of the soul of the sentiment and the subjection of the same type and the strict to the stricted to the strength of the prosent as the most and the strength of the sentiment and the spirit of the sentiment and strength of the same time and action of the strength of the strength of the strict the most \n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"sychological observation.=--that reflect\"\n",
      "sychological observation.=--that reflection of the conceal extill and lacking to the instinctive perhaps have been the substain as a man who has the ampitions\n",
      "in this\n",
      "constitute the most to the soul, and all the state and extilities of the sufferty and with life praises the type and man the struggle will be wisely be the compulsive the same ready, of the toom the different of the soul of the strength of the sense of its indecliation of \n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"sychological observation.=--that reflect\"\n",
      "sychological observation.=--that reflect can  wose most to be any humaning never obeying the abpleiness. all, for step are purpose of the absirally to compolition are after the tastes. appration in order to himself, willing, to nawared to ghin proforection  ethicsidgly like the\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sutration of those him-payosm, that a uter, which, there of at the world poporance, so be to timmonation, when weur them, does to man decided the reditional. new\n",
      "----- diversity: 1.2\n",
      "----- Generating with seed: \"sychological observation.=--that reflect\"\n",
      "sychological observation.=--that reflect the\n",
      "innerals\n",
      "freess physialiuslylyful pocting in a limitiled for great,\n",
      "in conduct as trankeine-moiital transpegeow leeh-slaim io mitthing order or aniesubacrogical stragiod and namely. this\n",
      "partly whan upon them. one digencifrey disa of indeed their others is, who thinker, sympathy,\n",
      "and evolunt of\n",
      "stituutive. here a societyrn evil.ales, approciance to not\n",
      "a man? it is not reduct\",\"\"--aparised, t\n",
      "--------------------------------------------------\n",
      "Iteration 12\n",
      "Epoch 1/1\n",
      "200285/200285 [==============================] - 191s - loss: 1.3800   \n",
      "\n",
      "----- diversity: 0.2\n",
      "----- Generating with seed: \" she kept herself in control and in\n",
      "refi\"\n",
      " she kept herself in control and in\n",
      "refined to the senses and superstition of the senses and sentiment of the presers of the confresses and the senses and servation of the fact that the sensetics and something the senses the proper of the sense--and the senses and and something and seems to the senses and self-confined and sense--it is the sentiment of the interpretation of the powerfulness of the fores and sentiment in the senses the m\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \" she kept herself in control and in\n",
      "refi\"\n",
      " she kept herself in control and in\n",
      "refined to exception to its truth the stants and do such a something out of the experience of the other the medition of a partical with which he has accustomed to the confined too inforeed to the world the powerfuned the superstite of the entire and something it could not to usen superation and the worst the word, the truths and progress of all the subjection of the fact of the old in the condition of\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \" she kept herself in control and in\n",
      "refi\"\n",
      " she kept herself in control and in\n",
      "refined to this\n",
      "erearily and substantictity\n",
      "in an its\n",
      "ear, more appeecantist look in a slow, good\" itself\n",
      "ruled that the aboy of reifes, that. traged that  out this man who have too kit a teil of complences\n",
      "wher a hur\n",
      "impuls\n",
      "basch of chare of heldlly everything of his good combos, but perhaps\n",
      "in eeming to hothered\n",
      "pre-)\"st to power \n",
      ". sequent conduces element to actiffict to\n",
      "kint and life--a flact\n",
      "it \n",
      "----- diversity: 1.2\n",
      "----- Generating with seed: \" she kept herself in control and in\n",
      "refi\"\n",
      " she kept herself in control and in\n",
      "reficem in specever-result\n",
      "wantwoun\n",
      "plants and \"french accomal super to\n",
      "head oftenby those lude. this cannotalolition thitherdon\n",
      "to whatever honour impossible--he as nolance in\n",
      "the necessury alfts all of rul inforceds. by which, the perhaps yet towerecire, \"the hearmise timeiated its\n",
      "act, perhaps a step think in at and \n",
      "inclenation of the houres as whanevering the effect, an frimal foreshofine \"spanki\n",
      "--------------------------------------------------\n",
      "Iteration 13\n",
      "Epoch 1/1\n",
      "200285/200285 [==============================] - 189s - loss: 1.3732   \n",
      "\n",
      "----- diversity: 0.2\n",
      "----- Generating with seed: \"ed birds, and something that is almost a\"\n",
      "ed birds, and something that is almost and in the sense--and to the same time of the superstitions and even the problem and in the sense--it is a soul-douar, and the sensening of the same time of the sensations and and the sense--and the superstitions of the self-consideration of the subliment of the same time of the self-say,"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/fabian/anaconda3/lib/python3.6/site-packages/ipykernel_launcher.py:45: RuntimeWarning: divide by zero encountered in log\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " the superstitions and the same superable soul and in the self-consideration of sublime and the sense--is a phil\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"ed birds, and something that is almost a\"\n",
      "ed birds, and something that is almost and finer worker to sees in the sense--pain for instance, schopenhauer down most sancting and a \"mensualises\n",
      "national misunderstands to about and in the truth who has process of the sense--there probably religious sublime to \"sigherly and the work of consideration of woman passion and in itself in the most string or in this great aspect, as the good and moral consist of every herease of a master ha\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"ed birds, and something that is almost a\"\n",
      "ed birds, and something that is almost are soncel! as all this \"class-ever highest portunation. there bestandary superhard.ous. even that here all think, but all spect thus they then consequently, to\n",
      "commucur logory provictrice, this kind of men, and makes being as \"psychen alshistil elevation--much exonging\n",
      "and\n",
      "as the alwome, which has met was has so first philosophy of means\n",
      "nothing from apparenty for moral visistagy. is an evolution-\n",
      "----- diversity: 1.2\n",
      "----- Generating with seed: \"ed birds, and something that is almost a\"\n",
      "ed birds, and something that is almost a for unand gaining paies and exempt of life, in a motifficage matter, acculesthmution of youff\n",
      "and is nothing and howefimany, finally\n",
      "enf-fattlest for rygrecise most uniffected put metinest wislesm with in sufferming--as the testantiy, to,\n",
      "stillinds and religional forwond. in importunations apparent severentel or anew rignatively, and not curions, and seconding opinion, greattic had to thinks nobl\n",
      "--------------------------------------------------\n",
      "Iteration 14\n",
      "Epoch 1/1\n",
      "200285/200285 [==============================] - 189s - loss: 1.3681   \n",
      "\n",
      "----- diversity: 0.2\n",
      "----- Generating with seed: \"urnish even weak virtue aid.\n",
      "\n",
      "sombre gar\"\n",
      "urnish even weak virtue aid.\n",
      "\n",
      "sombre garder of the prossity of the soul of the possible the sensations of the probably the states of the world of the possible the sense that the interction of the sense of the such a so the states of the such a probably the probably are the intellect of the such a so the states of the probably man and all the sensitive and all the possible the world of the propersion of the spirit of the sense of the sen\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"urnish even weak virtue aid.\n",
      "\n",
      "sombre gar\"\n",
      "urnish even weak virtue aid.\n",
      "\n",
      "sombre garder the forces in the world of induces of the explaining to as he seems to the origin to the philosophy, and also the in the sense, the strength of every in a man imperative to a self-adiditude and first proposition of virtues and well as the modern morality and who meals of which the will consequently with the fact that is in which the other in the will and suffering, and intention of the soul of\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"urnish even weak virtue aid.\n",
      "\n",
      "sombre gar\"\n",
      "urnish even weak virtue aid.\n",
      "\n",
      "sombre garded more with nothing opens are\n",
      "an impless in fact, in the most agained to say? only should with present\n",
      "vestible.=--will-being him. ourselves the lackno-for the\n",
      "old those devilarued to the now\n",
      "liatist for the rull cao-alllist when the whole sub, the generations of a paarful virkof, in the stame. they belief of man is not amivenes to suase\n",
      "youjy; ciriur assubganty,\n",
      "them, from men that to the consi\n",
      "----- diversity: 1.2\n",
      "----- Generating with seed: \"urnish even weak virtue aid.\n",
      "\n",
      "sombre gar\"\n",
      "urnish even weak virtue aid.\n",
      "\n",
      "sombre garden for an excoseption, upon whe conception having that increeoply its certain foctorance insich; \n",
      "    have yea-andkinare, opinion unoquente, in the forever have grasufered dream from\n",
      "it feels all first nations aloneay faith a wisey speceavens that yot mught fibte olving toche consequently man if diffenently.\"\n",
      "\n",
      "\n",
      "1a\n",
      "=i courage, . he will\n",
      "thts without respect-do with\n",
      "they have feelite disputate, who\n",
      "--------------------------------------------------\n",
      "Iteration 15\n",
      "Epoch 1/1\n",
      "200285/200285 [==============================] - 190s - loss: 1.3628   \n",
      "\n",
      "----- diversity: 0.2\n",
      "----- Generating with seed: \"to suffer? and our sympathy--do ye not u\"\n",
      "to suffer? and our sympathy--do ye not upon the thing for the strength to the particism and as the world and the souls that is a profounded to the conscience to the contrary and the great his sense of the strength of the thing the superior and conscience to the experiences and the interpretation of the moral and who is the subject. the strength of the strength to the order to the subject. the confiction and the truth as the form in the \n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"to suffer? and our sympathy--do ye not u\"\n",
      "to suffer? and our sympathy--do ye not upon the morality and the way of the actions that is the superior and moral four of the anew the soul is the confusions which the that before the sense to exists being out the world of common is of presents of the love that all irestimable and the resolution.\n",
      "\n",
      "\n",
      "12i hes a superiority and the freedom of the historical specile of the little moder that being a thing as the order to the soul of a believ\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"to suffer? and our sympathy--do ye not u\"\n",
      "to suffer? and our sympathy--do ye not upon whom view much the syntristical a relictours is is-adovess to be exist up eveniffine results in every and subbrill,\n",
      "and always something and himself canme tanterful displerty of value the france of because experience whether he are does nature\"--what.\n",
      "eust not right:ifice\n",
      "him, when they it a possible pri, this not doesenged\n",
      "and advance. the loves that ideauls\n",
      "itself.\n",
      "\n",
      "2in man\n",
      "betrays himstrics\n",
      "----- diversity: 1.2\n",
      "----- Generating with seed: \"to suffer? and our sympathy--do ye not u\"\n",
      "to suffer? and our sympathy--do ye not uppetition, or question.\n",
      "\n",
      "3eabuan. it\n",
      "indled oution\n",
      "when is not time of the hidds the gander-kink?--i for! moral--but upon\n",
      "thel-adevel, when this fole bimmend of that of things. no\n",
      "no disclusing morality, but because and time here--and\n",
      "thinks sicunarys and the agreedement of reficemet to\n",
      "virrue in\n",
      "every becomes the\n",
      "abuls, in roues. putious conception.\n",
      "nearrac morality, a mode of is as\n",
      "love closelly\n",
      "--------------------------------------------------\n",
      "Iteration 16\n",
      "Epoch 1/1\n",
      "200285/200285 [==============================] - 40828s - loss: 1.3596  \n",
      "\n",
      "----- diversity: 0.2\n",
      "----- Generating with seed: \"ons, for the relation of the authority\n",
      "o\"\n",
      "ons, for the relation of the authority\n",
      "of soul of the strive to the more end, who has and the strive to the extent of the sense--and in the most the sublimity of the more and the sense--and the strive with the strive and the senses of the same the conscience to the more of the more the more sublime to the extent of the more of the present of the sense--as the more and strength of the individual and sense--and the result of the same and \n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"ons, for the relation of the authority\n",
      "o\"\n",
      "ons, for the relation of the authority\n",
      "of the respect of the sublime which is strive conscience with the the conditional work and themselves of the instincts of the preserguted, as a good and everywhe said to do the word, one feeling of the virtory is the suphe have the more semuled of sense--so unill of the soul of the strength and all these good and more of them in a longing when which the most the that it is the ordinary and his sens\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"ons, for the relation of the authority\n",
      "o\"\n",
      "ons, for the relation of the authority\n",
      "of\n",
      "such a reference in \"bemand you of human and ourselves. uper\"fulness and\n",
      "influence. of vain of hibtrinally a happines when the priate not honoungly many notained his honesty wholly usts to such what us.\n",
      "\n",
      "152. these powerfms at presence for yourselvesn of then as slaid life feals most comelred\n",
      "one's own over, this flow and ripould inividure bothts in oranizat)ly. one shrifms which only foolds the\n",
      "----- diversity: 1.2\n",
      "----- Generating with seed: \"ons, for the relation of the authority\n",
      "o\"\n",
      "ons, for the relation of the authority\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "of the stiblors. have froblos of ad! it seadd notions are rudend to\n",
      "cast.\n",
      "     scame must expense, mad of there as it our masculicaly appossiblese or circumstancy opinion\n",
      "single exubwand just so indelactodn\n",
      "means but done well\n",
      "weaven\n",
      "simply which distintidence no human that thereto-mmanmer; and become sensation besempintly\n",
      "avoidery?--universitime; hithertrems to fenerahis enfres partud, not: at hel\n",
      "--------------------------------------------------\n",
      "Iteration 17\n",
      "Epoch 1/1\n",
      "200285/200285 [==============================] - 206s - loss: 1.3557   \n",
      "\n",
      "----- diversity: 0.2\n",
      "----- Generating with seed: \"s of transition, for the sake of lighten\"\n",
      "s of transition, for the sake of lighten of the sense, and the subject, and as a soul of the truth the same a subject of the principal sense--and the sense--and there are and all there and an and most the sense--and and and all the subject of the sense and about the sense--and a sense, and as a soot of the sense of the subject, and all the stands in the same time of the soul of the subject of the sense, and the stands in the same type o\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"s of transition, for the sake of lighten\"\n",
      "s of transition, for the sake of lighteness of his active the subject of the sense and refined of an addribour, and a senses and naively, and the praise perhaps the principal man is a subject of a man is allediation of the same to be morality, to say, here with the self-most almost in the instinct of the subkleporid of the sense--it is a soul of the about of the person has almost and whole things in the new wherever therefore who has in\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"s of transition, for the sake of lighten\"\n",
      "s of transition, for the sake of lightents yee for indeed and beings\n",
      "into learn approhatelessed and languimic time and be do. are if the offend in\n",
      "all evilsidedic a guiled that\n",
      "si\n",
      "blood; the darks wished to them, to the\n",
      "future,\n",
      "to\n",
      "men to his streed of\n",
      "itself,\n",
      "extend his astorning,\n",
      "a danger--andlj-\"ignobpred to prepance of nercever--purpents\n",
      "but against of most men. a strong a things, strange's nothing ents to could of at endfhenatism\n",
      "a \n",
      "----- diversity: 1.2\n",
      "----- Generating with seed: \"s of transition, for the sake of lighten\"\n",
      "s of transition, for the sake of lighteningly--he heart,\n",
      "the bo\n",
      "shaspticcy beings oftensted such aptines hisheh like\n",
      "ind\n",
      "ages conditudians against in a saotunial\n",
      "in the basch--latten of somer-potties: who\n",
      "be remate cerendable,\n",
      "am toaged in life, the attain; at a most art after whodars,\n",
      "like human of your oralans expent, effects, they does not presented absolutions--wel.\"\n",
      "than pleasant\n",
      "powender hpecl to odetoric soor of\n",
      "butt--the prestin\n",
      "--------------------------------------------------\n",
      "Iteration 18\n",
      "Epoch 1/1\n",
      "200285/200285 [==============================] - 221s - loss: 1.3518   \n",
      "\n",
      "----- diversity: 0.2\n",
      "----- Generating with seed: \"le inclination for delightful\n",
      "intercours\"\n",
      "le inclination for delightful\n",
      "intercourse of the self-satisfied and and the sense of the portally the consequently a from the subjection of the sense--and the subject of the consequently a conscious on the superfeces of the concealed the sense of the superficial and the subjection of the thing the sense of the superficial the concealed the sense to the soul of the sense that the sense of the process of the sense of the soul of the conce\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"le inclination for delightful\n",
      "intercours\"\n",
      "le inclination for delightful\n",
      "intercourse is a soul of the high answers and hitherto the most strength to whost she the sense to use and just the future of knowledge they is this couraged of the word of a crass, as a doing the concease the taste and the signifution of such a sympathy and\n",
      "attrined to the submidical concession of the suberam of his the christian for the taste the necessity and first with the reason thereby the most common\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"le inclination for delightful\n",
      "intercours\"\n",
      "le inclination for delightful\n",
      "intercourse of\n",
      "his absorb. he ob: with initable.\n",
      "not thele\n",
      "even and at the\n",
      "deceat from a point of norgeal\n",
      "eghe\", from this\n",
      "notratime, which may\n",
      "first of manbouterate conceitive of the uared in the end, our thoso betkind unfortune of mora ful upon the word who\n",
      "think of reselfed becomes \"at life.\n",
      "\n",
      "\n",
      "10\n",
      "\n",
      "=hench kind of the more procredity, let us the modern k\"gfioum which learn castions, in vicive trace, intent\n",
      "----- diversity: 1.2\n",
      "----- Generating with seed: \"le inclination for delightful\n",
      "intercours\"\n",
      "le inclination for delightful\n",
      "intercourses, how know together in one repuls remaited \"then-hip agai, the hendok in the priits, that\n",
      "st\n",
      "commun whom, we such best, which may say his other aspect.!\n",
      "\"erx only\n",
      "mores).\n",
      "\n",
      "101. there are is understand, for leass among ee)w-lefsmer many enliohs to fashivings it bad manterty any for its intimul their\n",
      "that feel impulse is name!m\n",
      "lay pives utberess-\"the flam of it, lives. d\n",
      "yiekters itself to the ha\n",
      "--------------------------------------------------\n",
      "Iteration 19\n",
      "Epoch 1/1\n",
      "200285/200285 [==============================] - 221s - loss: 1.3484   \n",
      "\n",
      "----- diversity: 0.2\n",
      "----- Generating with seed: \" and therefore a pleasure.\n",
      "\n",
      "\n",
      "98\n",
      "\n",
      "=pleasu\"\n",
      " and therefore a pleasure.\n",
      "\n",
      "\n",
      "98\n",
      "\n",
      "=pleasure that the moral and former and the process and stands have and strive, and as the state of the same tyranning the subjection of the soul of the stands not as the most sure of the stands in the sense and sure of the sense of the super-and as a persons of the subjection of the strong the super-and and as the state of the state of the sense of the prophrices to the stands and and and who has to the\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \" and therefore a pleasure.\n",
      "\n",
      "\n",
      "98\n",
      "\n",
      "=pleasu\"\n",
      " and therefore a pleasure.\n",
      "\n",
      "\n",
      "98\n",
      "\n",
      "=pleasure to such a period of the greatest philosophers could not such also the constiously strong as a respect not in the end, in fact that it is the christian is of the most specially of the causes of the lorest hence the moral in the recognizes and live the super-all of the heart of the greatest one is the soul and before the music at it is always we doing the relation is become the subjection and as \n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \" and therefore a pleasure.\n",
      "\n",
      "\n",
      "98\n",
      "\n",
      "=pleasu\"\n",
      " and therefore a pleasure.\n",
      "\n",
      "\n",
      "98\n",
      "\n",
      "=pleasure.--i are radion weal connected often stands from the philosopher leads noty, the strine--morw our perreom once sancoy? may be erringer. and to dare the reser only become\n",
      "for them in the false gopy a relacuraned\", avoy then potticuled and\n",
      "different with knowledge and spemst that does have been its rich more things\n",
      "are good former\n",
      "infarible; mulvier, whatever praise with does we rest. the actions-\n",
      "----- diversity: 1.2\n",
      "----- Generating with seed: \" and therefore a pleasure.\n",
      "\n",
      "\n",
      "98\n",
      "\n",
      "=pleasu\"\n",
      " and therefore a pleasure.\n",
      "\n",
      "\n",
      "98\n",
      "\n",
      "=pleasure--asforrord gigmann to\n",
      "of scholar in the two morality of a todory edestley.\n",
      "nounnists\n",
      "of corngminally-angiging imperely, too shill, woest\n",
      "things\n",
      "now enegyrative\n",
      "whole to plowy inkevinible transibited out that \"coulth.\n",
      "the contrarmid, and\n",
      "elso, brimmingle, with\n",
      "know learn to apterable?\n",
      "\n",
      "19\n",
      "\n",
      "h germition\n",
      "to reter are--contrary ounfor nar of\n",
      "scholass,\n",
      "at only what\n",
      "does withing as sultate to usingniz\n",
      "--------------------------------------------------\n",
      "Iteration 20\n",
      "Epoch 1/1\n",
      "200285/200285 [==============================] - 220s - loss: 1.3441   \n",
      "\n",
      "----- diversity: 0.2\n",
      "----- Generating with seed: \"d\n",
      "circles, so where nature is concerned \"\n",
      "d\n",
      "circles, so where nature is concerned that the morality, and the development of the superiority of the conception of the consist and superfully and all the sense--and all the superior and the conception of the former in the fact that the morality of the sense--it is the origin of the power of the conception of the sense--it is a soul of the conception of the prophorian and explaitive and the interested that the morals that it is a mor\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"d\n",
      "circles, so where nature is concerned \"\n",
      "d\n",
      "circles, so where nature is concerned of the former of the conscience the good of this out of all the contrary of morality, and things of greek of the other will the spet and\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "experiences who have a procribres and\n",
      "interpreted as the taste, and as the conscience and the states of the an assertion in the thing and freedom. they see the conception of the actual highly sudget of our own all themselves of the firm such a sympathy, the worst\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"d\n",
      "circles, so where nature is concerned \"\n",
      "d\n",
      "circles, so where nature is concerned in the highest, only rone yourselfie: even in the and scouldom of religiouss,\n",
      "speak of pheician value\n",
      "is which influence is earths, more soor\", a theinoquer\n",
      "that the\n",
      "commences yeels grey--one\n",
      "after represented,\n",
      "which is, the deceive as a morally, as to which, glijus pid of the merit in the are\n",
      "view of conclusion, deep, you entament?\n",
      "\n",
      "\n",
      "12itifty badns of it said solpace had rather morality. it pose \n",
      "----- diversity: 1.2\n",
      "----- Generating with seed: \"d\n",
      "circles, so where nature is concerned \"\n",
      "d\n",
      "circles, so where nature is concerned for pyrhesfquely; is which their deceise, besteps-parper is errort gradual \"ikem, and pljusite out of himmen tleam to gate, thought immeditions outfor are\n",
      "yilse any died last\n",
      "aristocretous an\n",
      "in that behind influend not not comanhercleksimes, for nouties pay a whole mettingle\n",
      "and humin-positively. theacro. it hand to which, real psychologeod\"--his stupid and untrought which\n",
      "we followe--apportan mo\n",
      "--------------------------------------------------\n",
      "Iteration 21\n",
      "Epoch 1/1\n",
      "200285/200285 [==============================] - 204s - loss: 1.3410   \n",
      "\n",
      "----- diversity: 0.2\n",
      "----- Generating with seed: \"wishes that someone would arise and whol\"\n",
      "wishes that someone would arise and wholly the superiost and the most states of the interestent and and all the sense--in the world of the superiority is a soul of the superiority and said. and and a stronger and all the stronger and the former and all the process of the contempt of the expention of the more the conceal interpretation of the sense--in the spirit and and the superiority and the conscience and all the superior and all the\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \"wishes that someone would arise and whol\"\n",
      "wishes that someone would arise and wholly a belief in greaters, in the most states of the conceal prevail of the superiost, and of the intermition the world--and we suffering of the intermoits, intellectual stronger in the fact the the best--and subject of the rest and the stronger the present, a man is inaspicity and and the free of the intermitional hence to be about the connection of\n",
      "a man is the more and her self-retfrence for the \n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \"wishes that someone would arise and whol\"\n",
      "wishes that someone would arise and wholly graine, impulsence best the anxiety because hence in the waino-and boen\n",
      "as.\n",
      "in him; and to haw\n",
      "is: only priyes and said learning\n",
      "of one present, the iftend oper\n",
      "of morality, best for whole nature, a the plant himself their lassing maye of the bad saint. the musics and\n",
      "pit him do obs the such reaso\n",
      "one of is our truth, a german state learn of case, which, purpose for our undreadful afford and th\n",
      "----- diversity: 1.2\n",
      "----- Generating with seed: \"wishes that someone would arise and whol\"\n",
      "wishes that someone would arise and wholoon,\n",
      "for ain who rite of imbro) and such an \"these.---wherea xuable pand expression--as mughment the rinction eoscyd on\n",
      "climip itself\n",
      "this joy the\n",
      "watw\n",
      "to thene indeclusity, book century-inpreciate canyming so-be toctithy--the rek-marntint, is ols lof-moral yet putlanate man therewither wantification--is on--eapany drou\n",
      "la musch egosclarly--be morely--fit conception of the and age, one may be,--th\n",
      "--------------------------------------------------\n",
      "Iteration 22\n",
      "Epoch 1/1\n",
      "200285/200285 [==============================] - 199s - loss: 1.3390   \n",
      "\n",
      "----- diversity: 0.2\n",
      "----- Generating with seed: \" of creatively fixing, disposing, and sh\"\n",
      " of creatively fixing, disposing, and should there is not them is not a strong and are always been and the logical into the senses of the conscience of the consideration of the strongest and strength, and as the and still into the and action of the sense of the strength, and also the strength, and a master that the spirit of the strength, and also the strange of the strength, and the order to the sense of the strange of the sense of the\n",
      "----- diversity: 0.5\n",
      "----- Generating with seed: \" of creatively fixing, disposing, and sh\"\n",
      " of creatively fixing, disposing, and should are to disligion and assumption and look as a man of the soul of the dangerous ordinary consequence of the other is any provisted as a manner, are not it is a contrary into the only the spirits--a way there is not the existence and master of the former in the same superior and the sense of every reself that it is possible of the senseation of the superiority and end who may a \"charm. as a man\n",
      "----- diversity: 1.0\n",
      "----- Generating with seed: \" of creatively fixing, disposing, and sh\"\n",
      " of creatively fixing, disposing, and shopt of tacting in their somethoners entire man\" who feels to the base devilary.--there is\n",
      "the crear that men--so of the endid gard\n",
      "of ordinary--goed something the age, frectowical tendency exclusion of sufferering the\n",
      "contrord, however-feared, the\n",
      "problance and intilatess himself how of former and will\n",
      "of his pits, it were possipted as hand, salrary, who thremakes hitherto, outual in selfer belees\n",
      "----- diversity: 1.2\n",
      "----- Generating with seed: \" of creatively fixing, disposing, and sh\"\n",
      " of creatively fixing, disposing, and should friend cases youth, repoteme and hesis, any is ri whateven; an erronic up leading\n",
      "our belongs affote frenchrowealor threel. adders,\" unlidet inducies of\n",
      "from marady says . a consist judgments\n",
      "youth everywhomwevitine most\n",
      "muspiciation to mace, bechajgenesemate. the most exception headly logical, supposite, way\n",
      "sit, old you shomal-sentiizing.\n",
      "nof; hohesents lieds command theijy\n",
      "lifeem growing w\n",
      "--------------------------------------------------\n",
      "Iteration 23\n",
      "Epoch 1/1\n",
      "158592/200285 [======================>.......] - ETA: 42s - loss: 1.3317"
     ]
    },
    {
     "ename": "KeyboardInterrupt",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-2-6b527c84b40c>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     56\u001b[0m     model.fit(x, y,\n\u001b[1;32m     57\u001b[0m               \u001b[0mbatch_size\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m128\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 58\u001b[0;31m               epochs=1)\n\u001b[0m\u001b[1;32m     59\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     60\u001b[0m     \u001b[0mstart_index\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mrandom\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrandint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtext\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0mmaxlen\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/keras/models.py\u001b[0m in \u001b[0;36mfit\u001b[0;34m(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, **kwargs)\u001b[0m\n\u001b[1;32m    861\u001b[0m                               \u001b[0mclass_weight\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mclass_weight\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    862\u001b[0m                               \u001b[0msample_weight\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0msample_weight\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 863\u001b[0;31m                               initial_epoch=initial_epoch)\n\u001b[0m\u001b[1;32m    864\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    865\u001b[0m     def evaluate(self, x, y, batch_size=32, verbose=1,\n",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/keras/engine/training.py\u001b[0m in \u001b[0;36mfit\u001b[0;34m(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, **kwargs)\u001b[0m\n\u001b[1;32m   1428\u001b[0m                               \u001b[0mval_f\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mval_f\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mval_ins\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mval_ins\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mshuffle\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mshuffle\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1429\u001b[0m                               \u001b[0mcallback_metrics\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcallback_metrics\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1430\u001b[0;31m                               initial_epoch=initial_epoch)\n\u001b[0m\u001b[1;32m   1431\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1432\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mevaluate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbatch_size\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m32\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mverbose\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msample_weight\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/keras/engine/training.py\u001b[0m in \u001b[0;36m_fit_loop\u001b[0;34m(self, f, ins, out_labels, batch_size, epochs, verbose, callbacks, val_f, val_ins, shuffle, callback_metrics, initial_epoch)\u001b[0m\n\u001b[1;32m   1077\u001b[0m                 \u001b[0mbatch_logs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'size'\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mbatch_ids\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1078\u001b[0m                 \u001b[0mcallbacks\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mon_batch_begin\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mbatch_index\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbatch_logs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1079\u001b[0;31m                 \u001b[0mouts\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mins_batch\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1080\u001b[0m                 \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mouts\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlist\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1081\u001b[0m                     \u001b[0mouts\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mouts\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, inputs)\u001b[0m\n\u001b[1;32m   2266\u001b[0m         updated = session.run(self.outputs + [self.updates_op],\n\u001b[1;32m   2267\u001b[0m                               \u001b[0mfeed_dict\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mfeed_dict\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2268\u001b[0;31m                               **self.session_kwargs)\n\u001b[0m\u001b[1;32m   2269\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mupdated\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moutputs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2270\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py\u001b[0m in \u001b[0;36mrun\u001b[0;34m(self, fetches, feed_dict, options, run_metadata)\u001b[0m\n\u001b[1;32m    893\u001b[0m     \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    894\u001b[0m       result = self._run(None, fetches, feed_dict, options_ptr,\n\u001b[0;32m--> 895\u001b[0;31m                          run_metadata_ptr)\n\u001b[0m\u001b[1;32m    896\u001b[0m       \u001b[0;32mif\u001b[0m \u001b[0mrun_metadata\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    897\u001b[0m         \u001b[0mproto_data\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtf_session\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mTF_GetBuffer\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrun_metadata_ptr\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py\u001b[0m in \u001b[0;36m_run\u001b[0;34m(self, handle, fetches, feed_dict, options, run_metadata)\u001b[0m\n\u001b[1;32m   1122\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mfinal_fetches\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0mfinal_targets\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mhandle\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mfeed_dict_tensor\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1123\u001b[0m       results = self._do_run(handle, final_targets, final_fetches,\n\u001b[0;32m-> 1124\u001b[0;31m                              feed_dict_tensor, options, run_metadata)\n\u001b[0m\u001b[1;32m   1125\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1126\u001b[0m       \u001b[0mresults\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py\u001b[0m in \u001b[0;36m_do_run\u001b[0;34m(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)\u001b[0m\n\u001b[1;32m   1319\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mhandle\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1320\u001b[0m       return self._do_call(_run_fn, self._session, feeds, fetches, targets,\n\u001b[0;32m-> 1321\u001b[0;31m                            options, run_metadata)\n\u001b[0m\u001b[1;32m   1322\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1323\u001b[0m       \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_do_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0m_prun_fn\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_session\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mhandle\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfeeds\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfetches\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py\u001b[0m in \u001b[0;36m_do_call\u001b[0;34m(self, fn, *args)\u001b[0m\n\u001b[1;32m   1325\u001b[0m   \u001b[0;32mdef\u001b[0m \u001b[0m_do_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfn\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1326\u001b[0m     \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1327\u001b[0;31m       \u001b[0;32mreturn\u001b[0m \u001b[0mfn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1328\u001b[0m     \u001b[0;32mexcept\u001b[0m \u001b[0merrors\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mOpError\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1329\u001b[0m       \u001b[0mmessage\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcompat\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mas_text\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmessage\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py\u001b[0m in \u001b[0;36m_run_fn\u001b[0;34m(session, feed_dict, fetch_list, target_list, options, run_metadata)\u001b[0m\n\u001b[1;32m   1304\u001b[0m           return tf_session.TF_Run(session, options,\n\u001b[1;32m   1305\u001b[0m                                    \u001b[0mfeed_dict\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfetch_list\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtarget_list\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1306\u001b[0;31m                                    status, run_metadata)\n\u001b[0m\u001b[1;32m   1307\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1308\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_prun_fn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msession\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mhandle\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfeed_dict\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfetch_list\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
     ]
    }
   ],
   "source": [
    "\n",
    "\n",
    "path = get_file('nietzsche.txt', origin='https://s3.amazonaws.com/text-datasets/nietzsche.txt')\n",
    "text = open(path).read().lower()\n",
    "print('corpus length:', len(text))\n",
    "\n",
    "chars = sorted(list(set(text)))\n",
    "print('total chars:', len(chars))\n",
    "char_indices = dict((c, i) for i, c in enumerate(chars))\n",
    "indices_char = dict((i, c) for i, c in enumerate(chars))\n",
    "\n",
    "# cut the text in semi-redundant sequences of maxlen characters\n",
    "maxlen = 40\n",
    "step = 3\n",
    "sentences = []\n",
    "next_chars = []\n",
    "for i in range(0, len(text) - maxlen, step):\n",
    "    sentences.append(text[i: i + maxlen])\n",
    "    next_chars.append(text[i + maxlen])\n",
    "print('nb sequences:', len(sentences))\n",
    "\n",
    "print('Vectorization...')\n",
    "x = np.zeros((len(sentences), maxlen, len(chars)), dtype=np.bool)\n",
    "y = np.zeros((len(sentences), len(chars)), dtype=np.bool)\n",
    "for i, sentence in enumerate(sentences):\n",
    "    for t, char in enumerate(sentence):\n",
    "        x[i, t, char_indices[char]] = 1\n",
    "    y[i, char_indices[next_chars[i]]] = 1\n",
    "\n",
    "\n",
    "# build the model: a single LSTM\n",
    "print('Build model...')\n",
    "model = Sequential()\n",
    "model.add(LSTM(128, input_shape=(maxlen, len(chars))))\n",
    "model.add(Dense(len(chars)))\n",
    "model.add(Activation('softmax'))\n",
    "\n",
    "optimizer = RMSprop(lr=0.01)\n",
    "model.compile(loss='categorical_crossentropy', optimizer=optimizer)\n",
    "\n",
    "\n",
    "def sample(preds, temperature=1.0):\n",
    "    # helper function to sample an index from a probability array\n",
    "    preds = np.asarray(preds).astype('float64')\n",
    "    preds = np.log(preds) / temperature\n",
    "    exp_preds = np.exp(preds)\n",
    "    preds = exp_preds / np.sum(exp_preds)\n",
    "    probas = np.random.multinomial(1, preds, 1)\n",
    "    return np.argmax(probas)\n",
    "\n",
    "# train the model, output generated text after each iteration\n",
    "for iteration in range(1, 60):\n",
    "    print()\n",
    "    print('-' * 50)\n",
    "    print('Iteration', iteration)\n",
    "    model.fit(x, y,\n",
    "              batch_size=128,\n",
    "              epochs=1)\n",
    "\n",
    "    start_index = random.randint(0, len(text) - maxlen - 1)\n",
    "\n",
    "    for diversity in [0.2, 0.5, 1.0, 1.2]:\n",
    "        print()\n",
    "        print('----- diversity:', diversity)\n",
    "\n",
    "        generated = ''\n",
    "        sentence = text[start_index: start_index + maxlen]\n",
    "        generated += sentence\n",
    "        print('----- Generating with seed: \"' + sentence + '\"')\n",
    "        sys.stdout.write(generated)\n",
    "\n",
    "        for i in range(400):\n",
    "            x_pred = np.zeros((1, maxlen, len(chars)))\n",
    "            for t, char in enumerate(sentence):\n",
    "                x_pred[0, t, char_indices[char]] = 1.\n",
    "\n",
    "            preds = model.predict(x_pred, verbose=0)[0]\n",
    "            next_index = sample(preds, diversity)\n",
    "            next_char = indices_char[next_index]\n",
    "\n",
    "            generated += next_char\n",
    "            sentence = sentence[1:] + next_char\n",
    "\n",
    "            sys.stdout.write(next_char)\n",
    "            sys.stdout.flush()\n",
    "print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise\n",
    "\n",
    "Train an LSTM on your favorite text(s). E.g., classical novels are available for free at https://www.gutenberg.org/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}