updated prior to read through forecast().

http://stackoverflow.com/questions/31409591/difference-between-forecast-and-predict-function-in-r?answertab=votes#tab-top

Author: englianhu

binary-forex-trading.Rmd

@@ -0,0 +1,91 @@
+---
+title: "Binary.com Forex Trading"
+subtitle: "[binary.com](https://github.com/englianhu/binary.com-interview-question) Interview Question"
+author: "[®γσ, Lian Hu](https://englianhu.github.io/) <img src='www/ENG.jpg' width='24'> <img src='www/RYO.jpg' width='24'>白戸則道®"
+date: "`r Sys.Date()`"
+output:
+  tint::tintHtml: 
+     self_contained: TRUE
+bibliography: skeleton.bib
+link-citations: yes
+---
+
+```{r setup, include = FALSE}
+suppressPackageStartupMessages(library('BBmisc'))
+suppressAll(library('tint'))
+suppressAll(library('devtools'))
+suppressAll(library('lubridate'))
+suppressAll(library('plyr'))
+suppressAll(library('stringr'))
+suppressAll(library('magrittr'))
+suppressAll(library('dplyr'))
+suppressAll(library('tidyr'))
+suppressAll(library('readr'))
+suppressAll(library('tidyverse')) #load c(dplyr, tidyr, stringr, readr) due to system doesn't work.
+suppressAll(library('DT'))
+suppressAll(library('quantmod'))
+suppressAll(library('formattable'))
+suppressAll(library('highcharter'))
+suppressAll(library('PerformanceAnalytics'))
+#'@ suppressAll(source('./function/.R', local = TRUE))
+
+# invalidate cache when the package version changes
+knitr::opts_chunk$set(tidy = FALSE, cache.extra = packageVersion('tint'))
+options(htmltools.dir.version = FALSE)
+```
+
+# 1. Introduction
+
+```{r read-data, cache = TRUE, echo = FALSE}
+#'@ m.pnorm <- list.files(path = './data/20150102', pattern = paste0('^wt.pnorm.fitgaum', 1:168, '.rds$'))
+#'@ m.phalfnorm <- list.files(path = './data/20150102', pattern = '^wt.phalfnorm.fitgaum[0-9]{1,}.rds$')
+fld <- list.files(path = './data', pattern = '^[0-9]{8}$')
+
+
+llply(fld[1], function(x) {
+    llply(list.files(path = paste0('./data/', x), pattern = '^wt.(phalfnorm|pnorm).fitgaum[0-9]{1,}.rds$'), function(y) {
+        read_rds(path = paste0('./data/', x, '/', y))
+    })
+  })
+
+```
+
+  I used to built the model and keen on the sportsbook trading. There is a good challenge for me to learn about the **Forex Trading** [binary.com Interview Qestion](https://github.com/englianhu/binary.com-interview-question). Here I try to apply the **Ridge**, **LASSO** and **Elasticnet Regularization** regression model as refer to [*Stock Market Forecasting Using LASSO Linear Regression Model*](https://github.com/englianhu/binary.com-interview-question/blob/master/reference/Stock%20Market%20Forecasting%20Using%20LASSO%20Linear%20Regression%20Model.pdf)^[1st paper in [Reference]].
+
+# 2. Content
+
+## 2.1 Modelling
+
+  I try to test 224 models and every single model has 11 sub models which adjusted with *alpha* value from *0, 0.1, 0.2... 1.0*. Due to the heavily loading of the data and will slow down the apps, here I try to create this Rmd file for models comparison purpose. You are feel free to browse over the [Webbase Application](https://beta.rstudioconnect.com/content/2367/).
+
+## 2.2 Comparison
+
+  When I .
+
+
+# 3. Conclusion
+
+
+
+## Reference
+
+  01. [Stock Market Forecasting Using LASSO Linear Regression Model](https://github.com/englianhu/binary.com-interview-question/blob/master/reference/Stock%20Market%20Forecasting%20Using%20LASSO%20Linear%20Regression%20Model.pdf)<img src='www/hot.jpg' width='20'>
+  02. [Using LASSO from lars (or glmnet) package in R for variable selection](http://stats.stackexchange.com/questions/58531/using-lasso-from-lars-or-glmnet-package-in-r-for-variable-selection?answertab=votes#tab-top)
+  03. [Difference between glmnet() and cv.glmnet() in R?](Difference between glmnet() and cv.glmnet() in R?)
+  04. [Testing Kelly Criterion and Optimal f in R](https://alphaism.wordpress.com/2012/04/13/testing-kelly-criterion-and-optimal-f-in-r) <img src='www/hot.jpg' width='20'>
+  05. [Portfolio Optimization and Monte Carlo Simulation](https://github.com/scibrokes/kelly-criterion/blob/master/references/Portfolio%20Optimization%20and%20Monte%20Carlo%20Simulation.pdf) <img src='www/hot.jpg' width='20'>
+  06. [Glmnet Vignette](https://web.stanford.edu/~hastie/glmnet/glmnet_alpha.html)
+  07. [lasso怎么用算法实现？](http://cos.name/cn/topic/101533/#post-418215)
+  08. [The Sparse Matrix and {glmnet}](http://amunategui.github.io/sparse-matrix-glmnet/)
+  09. [Regularization and Variable Selection via the Elastic Net](https://github.com/englianhu/binary.com-interview-question/blob/master/reference/Regularization%20and%20Variable%20Selection%20via%20the%20Elastic%20Net.pdf)
+  10. [LASSO, Ridge, and Elastic Net](http://www4.stat.ncsu.edu/~post/josh/LASSO_Ridge_Elastic_Net_-_Examples.html) <img src='www/hot.jpg' width='20'>
+  11. [热门数据挖掘模型应用入门（一）: LASSO回归](http://cos.name/2016/10/data-mining-1-lasso/)
+  12. [The Lasso Page](http://statweb.stanford.edu/~tibs/lasso.html)
+  13. [Call_Valuation.R](https://api.rpubs.com/Mariano/call)
+  14. [Lecture 6 – Stochastic Processes and Monte Carlo](http://zorro-trader.com/manual/en/Lecture%206.htm)
+  15. [The `caret` Package](http://topepo.github.io/caret/index.html) <img src='www/hot.jpg' width='20'>
+  16. [Time Series Cross Validation](https://rpubs.com/crossxwill/time-series-cv) <img src='www/hot.jpg' width='20'>
+  17. [Character-Code.com](http://character-code.com/)
+  18. [Size Matters – Kelly Optimization](https://alphaism.wordpress.com/2012/03/26/size-matters-kelly-optimization/) <img src='www/hot.jpg' width='20'>
+
+**Powered by - Copyright® Intellectual Property Rights of <img src='figure/oda-army.jpg' width='24'> [Scibrokes®](http://www.scibrokes.com)個人の経営企業**

binary-forex-trading.html

@@ -179,7 +179,7 @@ compStocks <- function(mbase, family = 'gaussian', xy.matrix = c('h1', 'h2'),
     gm <- paste(paste0(
       nam, seq(gaum), " <- ", gaum, 
       "; if(.save == TRUE) saveRDS(", nam, seq(gaum), ", file = '", pth, "/", nam, seq(gaum), ".rds')", 
-      "; if(.print == TRUE) cat('gaussian model ", pth, seq(gaum), "/", length(gaum), " ", txt, ".\n')"), 
+      "; if(.print == TRUE) cat('gaussian model ", pth, ' : ', as.character(now()), ' : ', seq(gaum), "/", length(gaum), " ", txt, ".\n')"), 
       collapse = "; ")
 
     ## start algorithmic calculation.
@@ -287,7 +287,7 @@ compStocks <- function(mbase, family = 'gaussian', xy.matrix = c('h1', 'h2'),
     bm <- paste(paste0(
       nam, seq(binm), " <- ", binm, 
       "; if(.save == TRUE) saveRDS(", nam, seq(binm), ", file = '", pth, "/", nam, seq(binm), ".rds')", 
-      "; if(.print == TRUE) cat('binomial model ",  pth, seq(binm), "/", length(binm), " ", txt, ".\n')"), 
+      "; if(.print == TRUE) cat('binomial model ",  pth, ' : ', as.character(now()), ' : ', seq(binm), "/", length(binm), " ", txt, ".\n')"), 
       collapse = "; ")
 
     ## start algorithmic calculation.
@@ -384,7 +384,7 @@ compStocks <- function(mbase, family = 'gaussian', xy.matrix = c('h1', 'h2'),
     pm <- paste(paste0(
       nam, seq(poim), " <- ", poim, 
       "; if(.save == TRUE) saveRDS(", nam, seq(poim), ", file = '", pth, "/", nam, seq(poim), ".rds')", 
-      "; if(.print == TRUE) cat('poisson model ",  pth, seq(poim), "/", length(poim), " ", txt, ".\n')"), 
+      "; if(.print == TRUE) cat('poisson model ",  pth, ' : ', as.character(now()), ' : ', seq(poim), "/", length(poim), " ", txt, ".\n')"), 
       collapse = "; ")
 
     ## start algorithmic calculation.
@@ -485,7 +485,7 @@ compStocks <- function(mbase, family = 'gaussian', xy.matrix = c('h1', 'h2'),
     mm <- paste(paste0(
       nam, seq(mnmm), " <- ", mnmm, 
       "; if(.save == TRUE) saveRDS(", nam, seq(mnmm), ", file = '", pth, "/", nam, seq(mnmm), ".rds')", 
-      "; if(.print == TRUE) cat('multinomial model ",  pth, seq(mnmm), "/", length(mnmm), " ", txt, ".\n')"), 
+      "; if(.print == TRUE) cat('multinomial model ",  pth, ' : ', as.character(now()), ' : ', seq(mnmm), "/", length(mnmm), " ", txt, ".\n')"), 
       collapse = "; ")
 
     ## start algorithmic calculation.
@@ -563,7 +563,7 @@ compStocks <- function(mbase, family = 'gaussian', xy.matrix = c('h1', 'h2'),
     cm <- paste(paste0(
       nam, seq(coxm), " <- ", coxm, 
       "; if(.save == TRUE) saveRDS(", nam, seq(coxm), ", file = '", pth, "/", nam, seq(coxm), ".rds')", 
-      "; if(.print == TRUE) cat('multinomial model ",  pth, seq(coxm), "/", length(coxm), " ", txt, ".\n')"), 
+      "; if(.print == TRUE) cat('multinomial model ",  pth, ' : ', as.character(now()), ' : ', seq(coxm), "/", length(coxm), " ", txt, ".\n')"), 
       collapse = "; ")
 
     ## start algorithmic calculation.
@@ -641,7 +641,7 @@ compStocks <- function(mbase, family = 'gaussian', xy.matrix = c('h1', 'h2'),
     mgm <- paste(paste0(
       nam, seq(mgam), " <- ", mgam, 
       "; if(.save == TRUE) saveRDS(", nam, seq(mgam), ", file = '", pth, "/", nam, seq(mgam), ".rds')", 
-      "; if(.print == TRUE) cat('mgaussian model ",  pth, seq(mgam), "/", length(mgam), " ", txt, ".\n')"), 
+      "; if(.print == TRUE) cat('mgaussian model ",  pth, ' : ', as.character(now()), ' : ', seq(mgam), "/", length(mgam), " ", txt, ".\n')"), 
       collapse = "; ")
 
     ## start algorithmic calculation.

binary-forex-trading_files/figure-html/fig-fullwidth-1.png

@@ -679,6 +679,27 @@ wt <- ldply(files, function(x) {
   y %>% data.frame(Date = x, .) %>% tbl_df %>% filter(mse == min(mse))
 }) %>% tbl_df
 
+## =============================== Models Comparison =========================================
+## Comparison of models.
+
+## list.files('./data/20150102', pattern = '^wt.pnorm.fitgaum[0-9]{1,}.rds$')
+m.pnorm <- list.files('./data/20150102', pattern = '^wt.pnorm.fitgaum[0-9]{1,}.rds$')
+m.phalfnorm <- list.files('./data/20150102', pattern = '^wt.phalfnorm.fitgaum[0-9]{1,}.rds$')
+
+m.pnorm <- list.files('./data/20150102', pattern = paste0('^wt.pnorm.fitgaum', 1:168, '.rds$'))
+m.phalfnorm <- list.files('./data/20150102', pattern = '^wt.phalfnorm.fitgaum[0-9]{1,}.rds$')
+
+
+## http://stackoverflow.com/questions/31409591/difference-between-forecast-and-predict-function-in-r?answertab=votes#tab-top
+
+## =========================== start reminder ============================================
+## Read through above link and might use forecast() after xts() the fit$yhat with date.
+## However the weighted function and values need to be consider. If the forecast() fit the 
+##   weighted effect or not in default.
+
+## ============================ end reminder =============================================
+
+
 
 
 ## ==================================== MCMC Model ===========================================

binary-forex-trading_files/figure-html/fig-main-1.png

@@ -8,6 +8,7 @@ filterLAD <- memoise(function(startDate = NULL, endDate = NULL) {
   suppressAll(library('magrittr'))
   suppressAll(library('lubridate'))
   suppressAll(library('formattable'))
+  suppressAll(library('xts'))
   suppressAll(library('quantmod'))
   #'@ suppressAll(library('tidyquant'))
   suppressAll(source('./function/loadLAD.R'))
@@ -23,8 +24,9 @@ filterLAD <- memoise(function(startDate = NULL, endDate = NULL) {
   }
 
   if(!exists('mbaseDT')) {
-    mbaseDT <- LAD %>% data.frame %>% data.frame(Date = rownames(.), .) %>% 
-      tbl_df %>% mutate(Date = ymd(Date)) %>% arrange(Date)
+    mbaseDT <- data.frame(Date = index(LAD), coredata(LAD)) %>% tbl_df %>% arrange(Date)
+    #'@ mbaseDT <- LAD %>% data.frame %>% data.frame(Date = rownames(.), .) %>% 
+    #'@   tbl_df %>% mutate(Date = ymd(Date)) %>% arrange(Date)
     # %>% mutate(LAD.Volume = formattable::digits(
     #       LAD.Volume, 0, format = 'd', big.mark = ','))
   }
@@ -43,8 +45,9 @@ filterLAD <- memoise(function(startDate = NULL, endDate = NULL) {
     suppressAll(getSymbols('LAD', from = startDate, to = endDate))
 
     mbase <- LAD; rm(LAD)
-    mbaseDT <- mbase %>% data.frame %>% data.frame(Date = rownames(.), .) %>% 
-      tbl_df %>% mutate(Date = ymd(Date)) %>% arrange(Date)
+    mbaseDT <- data.frame(Date = index(LAD), coredata(LAD)) %>% tbl_df %>% arrange(Date)
+    #'@ mbaseDT <- mbase %>% data.frame %>% data.frame(Date = rownames(.), .) %>% 
+    #'@   tbl_df %>% mutate(Date = ymd(Date)) %>% arrange(Date)
     # %>% mutate(LAD.Volume = formattable::digits(
     #       LAD.Volume, 0, format = 'd', big.mark = ','))
     dateRange <- range(mbaseDT$Date)
@@ -58,8 +61,9 @@ filterLAD <- memoise(function(startDate = NULL, endDate = NULL) {
   ## ==================== Data Filtering ===================================
   mbase <- mbase[paste0(startDate, '/', endDate)]
 
-  mbaseDT <- mbase %>% data.frame %>% data.frame(Date = rownames(.), .) %>% 
-    tbl_df %>% mutate(Date = ymd(Date)) %>% arrange(Date)
+  mbaseDT <- data.frame(Date = index(LAD), coredata(LAD)) %>% tbl_df %>% arrange(Date)
+  #'@ mbaseDT <- mbase %>% data.frame %>% data.frame(Date = rownames(.), .) %>% 
+  #'@   tbl_df %>% mutate(Date = ymd(Date)) %>% arrange(Date)
   # %>% mutate(LAD.Volume = formattable::digits(
   #       LAD.Volume, 0, format = 'd', big.mark = ','))
 

binary-forex-trading_files/figure-html/fig-margin-1.png

@@ -9,6 +9,7 @@ loadLAD <- function(latestDate = NULL, dataSizeDay = 365) {
   suppressAll(library('tidyr'))
   suppressAll(library('readr'))
   suppressAll(library('tidyverse')) #load c(dplyr, tidyr, stringr, readr) due to system doesn't work.
+  suppressAll(library('xts'))
   suppressAll(library('quantmod'))
   #'@ suppressAll(library('tidyquant'))
 
@@ -49,8 +50,9 @@ loadLAD <- function(latestDate = NULL, dataSizeDay = 365) {
     LAD <- read_rds(path = './data/LAD.rds')
   }
 
-  LADDT <- LAD %>% data.frame %>% data.frame(Date = rownames(.), .) %>% 
-    tbl_df %>% mutate(Date = ymd(Date)) %>% arrange(Date)
+  LADDT <- data.frame(Date = index(LAD), coredata(LAD)) %>% tbl_df %>% arrange(Date)
+  #'@ LADDT <- LAD %>% data.frame %>% data.frame(Date = rownames(.), .) %>% 
+  #'@   tbl_df %>% mutate(Date = ymd(Date)) %>% arrange(Date)
   #mutate(LAD.Volume = formattable::digits(
   #       LAD.Volume, 0, format = 'd', big.mark = ','))
 

binary-forex-trading_files/figure-html/fig-margin-separate-1.png

@@ -9,6 +9,7 @@ plotChart2 <- function(Fund, type = 'multiple', event = NULL, event.dates = NULL
   suppressMessages(library('formattable'))
   suppressMessages(library('quantmod'))
   suppressMessages(library('highcharter'))
+  suppressMessages(library('xts'))
   suppressMessages(library('tidyverse'))
 
   ## --------------------- Data validation ------------------------------------------

binary-forex-trading_files/figure-html/fig-margin-separate-2.png

@@ -1,7 +1,10 @@
-simcompS <- function(mbase, family, weight.dist = 'none') {
+simcompS <- function(mbase, family, weight.dist = 'none', rm.sim = TRUE) {
   ## Draft simulate the function while I added-on some-more criterias and selections.
   ## Simulate 365 days dataset with Markov Chain.
   ##   using moving 365 days dataset and rerun the compStocks.
+  
+  ## rm.sim = TRUE will remove the files once calculated and saved.
+  
   suppressPackageStartupMessages(library("BBmisc"))
   suppressAll(library('devtools'))
   suppressAll(library('lubridate'))
@@ -12,12 +15,14 @@ simcompS <- function(mbase, family, weight.dist = 'none') {
   suppressAll(library('tidyr'))
   suppressAll(library('readr'))
   suppressAll(library('tidyverse')) #load c(dplyr, tidyr, stringr, readr) due to system doesn't work.
+  suppressAll(library('xts'))
   suppressAll(library('quantmod'))
   suppressAll(source('./function/compStocks.R'))
 
   if(is.xts(mbase)) {
-    mbase <- mbase %>% data.frame %>% data.frame(Date = rownames(.), .) %>% 
-      tbl_df %>% mutate(Date = ymd(Date)) %>% arrange(Date)
+    mbase <- data.frame(Date = index(mbase), coredata(mbase)) %>% tbl_df %>% arrange(Date)
+    #'@ mbase <- mbase %>% data.frame %>% data.frame(Date = rownames(.), .) %>% 
+    #'@   tbl_df %>% mutate(Date = ymd(Date)) %>% arrange(Date)
     #mutate(LAD.Volume = formattable::digits(
     #       LAD.Volume, 0, format = 'd', big.mark = ','))
   }
@@ -98,7 +103,7 @@ simcompS <- function(mbase, family, weight.dist = 'none') {
       wt.pnorm.fitgaum.form = wt.pnorm.fitgaum$formula1[str_replace_all(
         name514gs, 'wt.pnorm.fitgaum', '') %>% as.numeric]
       saveRDS(wt.pnorm.fitgaum.form, file = paste0(pth, '/wt.pnorm.fitgaum.form.rds'))
-      rm(fld, fitnum, fl, wtdt, wtpc, wt.pnorm.fitgaum, wt.pnorm.fitgaum.mse1, wt.pnorm.fitgaum.best, wt.pnorm.fitgaum.sum, wt.pnorm.fitgaum.form)
+      if(rm.sim == TRUE) rm(fld, fitnum, fl, wtdt, wtpc, wt.pnorm.fitgaum, wt.pnorm.fitgaum.mse1, wt.pnorm.fitgaum.best, wt.pnorm.fitgaum.sum, wt.pnorm.fitgaum.form)
 
     } else if(weight.dist == 'phalfnorm') {
 
@@ -153,7 +158,7 @@ simcompS <- function(mbase, family, weight.dist = 'none') {
       wt.phalfnorm.fitgaum.form = wt.phalfnorm.fitgaum$formula1[str_replace_all(
         name514gs, 'wt.phalfnorm.fitgaum', '') %>% as.numeric]
       saveRDS(wt.phalfnorm.fitgaum.form, file = paste0(pth, '/wt.phalfnorm.fitgaum.form.rds'))
-      rm(fld, fitnum, fl, wtdt, wtpc, wt.phalfnorm.fitgaum, wt.phalfnorm.fitgaum.mse1, wt.phalfnorm.fitgaum.best, wt.phalfnorm.fitgaum.sum, wt.phalfnorm.fitgaum.form)
+      if(rm.sim == TRUE) rm(fld, fitnum, fl, wtdt, wtpc, wt.phalfnorm.fitgaum, wt.phalfnorm.fitgaum.mse1, wt.phalfnorm.fitgaum.best, wt.phalfnorm.fitgaum.sum, wt.phalfnorm.fitgaum.form)
 
     } else if(weight.dist == 'none') {
 
@@ -180,7 +185,7 @@ simcompS <- function(mbase, family, weight.dist = 'none') {
       ## saved best model's formula.
       fitgaum.form = fitgaum$formula1[str_replace_all(name514gs, 'fitgaum', '') %>% as.numeric]
       saveRDS(fitgaum.form, file = paste0(pth, '/fitgaum.form.rds'))
-      rm(fld, fitnum, fl, wtdt, wtpc, fitgaum, fitgaum.mse1, fitgaum.best, fitgaum.sum, fitgaum.form)
+      if(rm.sim == TRUE) rm(fld, fitnum, fl, wtdt, wtpc, fitgaum, fitgaum.mse1, fitgaum.best, fitgaum.sum, fitgaum.form)
 
     } else {
       stop('Kindly select weight.dist = "pnorm" or weight.dist = "phalfnorm" or weight.dist = "none".')

binary-forex-trading_files/figure-html/fig-margin-together-1.png

@@ -21,6 +21,7 @@ suppressAll(library("shiny"))
 suppressAll(library("shinyjs"))
 suppressAll(library('shinyBS'))
 suppressAll(library('DT'))
+suppressAll(library('xts'))
 suppressAll(library('quantmod'))
 suppressAll(library('formattable'))
 suppressAll(library('highcharter'))
@@ -35,8 +36,9 @@ suppressAll(source('./function/plotChart2.R'))
 #'@ getSymbols('LAD')
 #'@ saveRDS(LAD, file = './data/LAD_full.rds')
 LAD <- read_rds(path = './data/LAD_full.rds')
-LADDT <- LAD %>% data.frame %>% data.frame(Date = rownames(.), .) %>% 
-  tbl_df %>% mutate(Date = ymd(Date)) %>% arrange(Date)
+LADDT <- data.frame(Date = index(LAD), coredata(LAD)) %>% tbl_df %>% arrange(Date)
+#'@ LADDT <- LAD %>% data.frame %>% data.frame(Date = rownames(.), .) %>% 
+#'@   tbl_df %>% mutate(Date = ymd(Date)) %>% arrange(Date)
 #mutate(LAD.Volume = formattable::digits(
 #       LAD.Volume, 0, format = 'd', big.mark = ','))
 

binary-forex-trading_files/figure-html/fig-margin-together-2.png

@@ -0,0 +1,15 @@
+@Manual{R-base,
+  title = {R: A Language and Environment for Statistical Computing},
+  author = {{R Core Team}},
+  organization = {R Foundation for Statistical Computing},
+  address = {Vienna, Austria},
+  year = {2016},
+  url = {https://www.R-project.org/},
+}
+@Manual{R-rmarkdown,
+  title = {rmarkdown: Dynamic Documents for R},
+  author = {JJ Allaire and Joe Cheng and Yihui Xie and Jonathan McPherson and Winston Chang and Jeff Allen and Hadley Wickham and Aron Atkins and Rob Hyndman},
+  year = {2016},
+  note = {R package version 1.3},
+  url = {https://CRAN.R-project.org/package=rmarkdown},
+}