Irish Poker Boards
Register Arcade FAQ Members List Calendar Mark Forums Read

Go Back   Irish Poker Boards > Community, Tech, and Feedback > Personal Logs
Reply
 
Thread Tools Display Modes
Old 29-04-17, 06:43   #1
Hitchhiker's Guide To...
Duncan F*****g Stewart
 
Hitchhiker's Guide To...'s Avatar
 
Join Date: Jan 2010
Location: South Cidre
Posts: 30,030
[i + 1] years and counting ... / Shut up and code!

Thought I'd start this to track my progress in becoming a Coding God.

Likely to be infrequently updated, but the overall thread might be interesting as a path to going from knowing no coding to an eventual plan of being high level competent.

My personal goal in order of chronological aim:

1. Get to very competent in python initially
2. Get to reasonably competent in machine learning techniques
3. Add some related coding skills - e.g. R
4. Do some research projects with the new skills
5. Do some fintech/investment startup work using the skills
6. Get a ridiculously wellpaid research position in a hedge fund and ditch the neckbeards of academia

Time to completion of the six steps: by September 2018

Lets see how that works out
Hitchhiker's Guide To... is offline   Reply With Quote
4 Thanks From:
Old 29-04-17, 06:45   #2
Hitchhiker's Guide To...
Duncan F*****g Stewart
 
Hitchhiker's Guide To...'s Avatar
 
Join Date: Jan 2010
Location: South Cidre
Posts: 30,030
Step 1.1: Have finished this coursera course: Learn to Program: The Fundamentals.

This was a brill course. Really well delivered. Went for the certified version (€46) as have seen on CVs a lot of people now including Coursera certified courses and those courses being accepted.

You start the course with nothing and end with great basic knowledge of how python works.
Hitchhiker's Guide To... is offline   Reply With Quote
Old 29-04-17, 06:50   #3
Hitchhiker's Guide To...
Duncan F*****g Stewart
 
Hitchhiker's Guide To...'s Avatar
 
Join Date: Jan 2010
Location: South Cidre
Posts: 30,030
Step 1.2: Have enrolled in the follow-on course: Learn to Program: Crafting Quality Code.

Step 1.3: Got this highly recommended book: Automate the Boring Stuff with Python.

Aim is to work through both of these by the end of May. The course brings you from basics of how Python works to actually programming. The book shows what seems to be some of the breadth of possibility of Python.

Think by the end of May should therefore be at the stage of being at the last stages of beginner.
Hitchhiker's Guide To... is offline   Reply With Quote
Old 29-04-17, 08:03   #4
shrapnel
chevauche moi sur le côté, Kylian
 
shrapnel's Avatar
 
Join Date: Jan 2010
Posts: 6,860
i have a soft copy of "Learn Python the hard way" by Zed Shaw if you want, which was recommended to me by Tar a couple of years ago.
shrapnel is offline   Reply With Quote
Old 29-04-17, 09:28   #5
Flushdraw
The Gamesmaster
 
Flushdraw's Avatar
 
Join Date: Jan 2010
Location: Swieqi, Sunny Malta
Posts: 22,640
gl Hitch. Hope you have better luck finishing it than this
Flushdraw is offline   Reply With Quote
Old 30-05-17, 07:33   #6
Hitchhiker's Guide To...
Duncan F*****g Stewart
 
Hitchhiker's Guide To...'s Avatar
 
Join Date: Jan 2010
Location: South Cidre
Posts: 30,030
Quote:
Originally Posted by Hitchhiker's Guide To... View Post
Step 1.2: Have enrolled in the follow-on course: Learn to Program: Crafting Quality Code.

Step 1.3: Got this highly recommended book: Automate the Boring Stuff with Python.

Aim is to work through both of these by the end of May. The course brings you from basics of how Python works to actually programming. The book shows what seems to be some of the breadth of possibility of Python.

Think by the end of May should therefore be at the stage of being at the last stages of beginner.
Got through that course fine. Not as good as the first one, but complimented the first one very well.

The book is really good. Up as far as the web scraping chapter.
Hitchhiker's Guide To... is offline   Reply With Quote
Old 30-05-17, 07:38   #7
Hitchhiker's Guide To...
Duncan F*****g Stewart
 
Hitchhiker's Guide To...'s Avatar
 
Join Date: Jan 2010
Location: South Cidre
Posts: 30,030
Next plans for June:

Step 1.3: Finish off Automate the Boring Stuff book

Step 1.4: Enrolled in this Michigan uni course: Introduction to Data Science in Python. Which is the first of five courses from Michigan on data analysis in Python. Great reviews for the course and think I've enough basic knowledge of Python at this stage.

Step 1.5: Will also do this follow-on course from Michigan in June. Applied Plotting, Charting & Data Representation in Python. That'll get me about ready to do a machine learning course in July.
Hitchhiker's Guide To... is offline   Reply With Quote
Old 30-05-17, 07:39   #8
Hitchhiker's Guide To...
Duncan F*****g Stewart
 
Hitchhiker's Guide To...'s Avatar
 
Join Date: Jan 2010
Location: South Cidre
Posts: 30,030
Quote:
Originally Posted by shrapnel View Post
i have a soft copy of "Learn Python the hard way" by Zed Shaw if you want, which was recommended to me by Tar a couple of years ago.
Downloaded this. Looks like a great system of learning, but see there's a Python 3 version coming out in July, so going to hang on for that and maybe use it as a bit of a refresher course.
Hitchhiker's Guide To... is offline   Reply With Quote
Old 30-05-17, 07:52   #9
Hitchhiker's Guide To...
Duncan F*****g Stewart
 
Hitchhiker's Guide To...'s Avatar
 
Join Date: Jan 2010
Location: South Cidre
Posts: 30,030
Step 4 is: Do some research projects with the new skills


Step 4.1: Write a code to retrieve newspaper articles from a database and then analyse the sentiment in those articles to come up with a determinant of pricing model for a car sharing website.

Started this during the month by linking up with a marketing guy to do some textual analysis using python. The idea is to see if general fear levels in a country influence what price someone charges for a seat in their car on a big car sharing website. The thought being that fear levels increase the price someone needs to share space with an unknown stranger. This will be presented at a fintech conference in December in Taiwan all going well, so gives a nice deadline to act as impetus and motivation.

My initial task in the project is to create a mini-program that collects data from a newspaper website. Its a painfully slow process of learning. I'm using selenium which opens a phantom browser and automates entering search terms in a search box and retreiving the results. Only in the middle of it now, with the following code. I'd estimate for each line below it has taken about an hour to write. The line to click the 'I accept the terms and conditions' button easily took about five hours!

Code:
# le figaro import from LexisNexus

from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver.common.keys import Keys
import time


# Login into LexisNexis

chromedriver = 'C:\\Users\\micha\\Dropbox\\Learning\\Python\\chromedriver.exe'
browser = webdriver.Chrome(chromedriver)

browser.get('https://www-nexis-com.dcu.idm.oclc.org')

username = browser.find_element_by_id("username")
password = browser.find_element_by_id("password")

username.send_keys("XXX")
password.send_keys("XXX")

browser.find_element_by_name("_eventId_proceed").click()

browser.find_element_by_xpath("/html/body/table/tbody/tr[2]/td/table/tbody/tr/td[2]/table/tbody/tr[4]/td[2]/table/tbody/tr/td/table/tbody/tr[1]/td/a[1]/img").click()

# Set up search

        # clear search window
browser.find_element_by_name("searchTerms1").clear()

        # set search dates
select1 = Select(browser.find_element_by_xpath('//*[@id="specifyDateDefaultStyle"]'))
select1.select_by_visible_text("Custom date")

browser.find_element_by_name("fromDate").clear()
browser.find_element_by_name("fromDate").send_keys("01/01/2014")
browser.find_element_by_name("toDate").clear()
browser.find_element_by_name("toDate").send_keys("01/01/2014")
        # need some way to generate dates in the above format and also to loop over those dates

        # enter Le Figaro as the source
browser.find_element_by_xpath('//*[@id="rightcolumnlarge"]/fieldset[4]/div[1]/div').click()
browser.find_element_by_id("selected_source").send_keys("Le Figaro Online")
time.sleep(3)
browser.find_element_by_id("selected_source").send_keys(Keys.DOWN)
browser.find_element_by_id("selected_source").send_keys(Keys.ENTER)
time.sleep(2)

        # exclude short articles
browser.find_element_by_name("excludeShortDocsChecked").click()

        # start search
browser.find_element_by_class_name("power_submit").click()

Need to add in the bits for looping over the dates next, and then the actual download data part.

Last edited by Hitchhiker's Guide To...; 30-05-17 at 08:12.
Hitchhiker's Guide To... is offline   Reply With Quote
Old 30-05-17, 08:05   #10
Hitchhiker's Guide To...
Duncan F*****g Stewart
 
Hitchhiker's Guide To...'s Avatar
 
Join Date: Jan 2010
Location: South Cidre
Posts: 30,030
Noticed theres a few web scraping packages: Selenium as above, BeautifulSoup, and Scrapy.

Is there a good place to really get into the pros and cons of each? I've read quite a few general comparisons, but nothing that delves into a proper level of depth.
Hitchhiker's Guide To... is offline   Reply With Quote
Old 30-05-17, 10:41   #11
DeadParrot
Just Resting
 
DeadParrot's Avatar
 
Join Date: Jan 2010
Posts: 19,364
Wow... selenium
That takes me back to my testing days
__________________
People say I should be more humble I hope they understand, they don't listen when you mumble
Get a shiny metal Revolut card!
https://revolut.com/r/jamesb8
DeadParrot is offline   Reply With Quote
Old 30-05-17, 10:53   #12
Hitchhiker's Guide To...
Duncan F*****g Stewart
 
Hitchhiker's Guide To...'s Avatar
 
Join Date: Jan 2010
Location: South Cidre
Posts: 30,030
Quote:
Originally Posted by DeadParrot View Post
Wow... selenium
That takes me back to my testing days
ha, read that in some places, that it had been appropriated from much more noble endevours.
Hitchhiker's Guide To... is offline   Reply With Quote
Old 02-07-17, 21:35   #13
Hitchhiker's Guide To...
Duncan F*****g Stewart
 
Hitchhiker's Guide To...'s Avatar
 
Join Date: Jan 2010
Location: South Cidre
Posts: 30,030
Quote:
Originally Posted by Hitchhiker's Guide To... View Post
Next plans for June:

Step 1.3: Finish off Automate the Boring Stuff book

Step 1.4: Enrolled in this Michigan uni course: Introduction to Data Science in Python. Which is the first of five courses from Michigan on data analysis in Python. Great reviews for the course and think I've enough basic knowledge of Python at this stage.

Step 1.5: Will also do this follow-on course from Michigan in June. Applied Plotting, Charting & Data Representation in Python. That'll get me about ready to do a machine learning course in July.
Ended up skipping step 1.3 as there was nothing much more needed from the rest of the book.

Did the course in Step 1.4 'Introduction to Data Science in Python', and its a fantastic course. Although I don't know if its my dumbassery or the pacing of the course, but the coursework seemed far too difficult and I'd be spending 10-20 hours on each assignment that was guided for 3-4 hours. Even still I mightily struggled.

I suppose thats what learning is all about, trying to find out the things you don't know.

I think maybe more python than just the two basic courses completed was needed before tackling this. A great course though.

As a result of that and a few other life things (see bbv ad infinitum) I didn't get the second course (step 1.5) finished and am just doing it now. From the first two classes of this second course it seems to take the material of the first course and assume that was piss easy, but hopefully the assignments themselves work out easier than the course lectures make it seem.
Hitchhiker's Guide To... is offline   Reply With Quote
Old 02-07-17, 21:40   #14
Hitchhiker's Guide To...
Duncan F*****g Stewart
 
Hitchhiker's Guide To...'s Avatar
 
Join Date: Jan 2010
Location: South Cidre
Posts: 30,030
Plans for July:

Step 1.5: Do this data visualisation course from Michigan in June. Applied Plotting, Charting & Data Representation in Python.

Step 1.6: Do the Michigan machine learning course on coursera. Applied Machine Learning in Python.

Might struggle to get the second course done given how tough the slog has turned into, and I've only evenings after everyone is asleep to work on these courses.
Hitchhiker's Guide To... is offline   Reply With Quote
Old 02-07-17, 21:55   #15
Hitchhiker's Guide To...
Duncan F*****g Stewart
 
Hitchhiker's Guide To...'s Avatar
 
Join Date: Jan 2010
Location: South Cidre
Posts: 30,030
Think then in August I might take a step back and do another course on the topics covered so far. Maybe this course 'Python for Data Science' on edX. Or maybe the Data Scientist with Python package on DataCamp.

Am just conscious that I'm kinda just about 'getting' all the material, but there's too many gaps in my understanding. This type of bedding down step back would then allow a big leap forward into the truely complex stuff afterwards.
Hitchhiker's Guide To... is offline   Reply With Quote
Thanks From:
Old 19-07-17, 21:19   #16
Hitchhiker's Guide To...
Duncan F*****g Stewart
 
Hitchhiker's Guide To...'s Avatar
 
Join Date: Jan 2010
Location: South Cidre
Posts: 30,030
Quote:
Originally Posted by Hitchhiker's Guide To... View Post
Plans for July:

Step 1.5: Do this data visualisation course from Michigan in June. Applied Plotting, Charting & Data Representation in Python.

Step 1.6: Do the Michigan machine learning course on coursera. Applied Machine Learning in Python.

Might struggle to get the second course done given how tough the slog has turned into, and I've only evenings after everyone is asleep to work on these courses.
Both of these done. Oh it was a struggle. The machine learning one in particular. I understood all the ideas, but it was just the code to implement them that was mindboggling.

Machine learning is so clearly the future of everything though. Its absolutely amazing in terms of its potential and how it has upended previous ways of looking at how we can know things.

I kinda feel after this course that I've just looked in the shop window and spotted the goodies I'd like to someday have. Beginning of the beginning really.
Hitchhiker's Guide To... is offline   Reply With Quote
Thanks From:
Old 19-07-17, 21:28   #17
Hitchhiker's Guide To...
Duncan F*****g Stewart
 
Hitchhiker's Guide To...'s Avatar
 
Join Date: Jan 2010
Location: South Cidre
Posts: 30,030
Step 1.7: Complete the Data Scientist with Python program on DataCamp.

Read a lot of great things about this. The general view is that its not particularly great for learning from scratch, but if you have a few relevant uni courses under your belt then its excellent for getting the training in and getting you on top of the technicalities.

This is going to take a while. Have only a few spare evenings in August, and September is the start of intensive teaching. The plan though is to get it finished by the end of September.

Also going to pick up this hotly awaited book when the second edition is released at the end of September. Python for Data Analysis. The current edition is pretty much the pandas bible, but its now five years old so way too dated.

After finishing this I should be at an upper-intermediate level in general python and pandas, and maybe somewhere in the middle of the beginning of machine learning.
Hitchhiker's Guide To... is offline   Reply With Quote
Old 19-07-17, 21:34   #18
Hitchhiker's Guide To...
Duncan F*****g Stewart
 
Hitchhiker's Guide To...'s Avatar
 
Join Date: Jan 2010
Location: South Cidre
Posts: 30,030
After that then it'll be jumping into the famous Stanford Machine Learning course taught by Andrew Ng. Hadn't done this course up to this point as its in MATLAB which is a whole new software to get on top of.

The course strength seems to be in explaining why we're doing all these techniques and how we can choose between the various testing choices.

Then it'll be just getting more and more advanced in the area. At a rough guess then it would be to be at an upper-intermediate level in machine learning by the end of 2017, and some level of decent expertise by the end of the first half of 2018.
Hitchhiker's Guide To... is offline   Reply With Quote
Old 19-07-17, 21:43   #19
Hitchhiker's Guide To...
Duncan F*****g Stewart
 
Hitchhiker's Guide To...'s Avatar
 
Join Date: Jan 2010
Location: South Cidre
Posts: 30,030
Machine learning also offers many fascinating new ways to design this.

Hitchhiker's Guide To... is offline   Reply With Quote
Thanks From:
Old 24-07-17, 07:03   #20
Hitchhiker's Guide To...
Duncan F*****g Stewart
 
Hitchhiker's Guide To...'s Avatar
 
Join Date: Jan 2010
Location: South Cidre
Posts: 30,030
Great article on the potential of machine learning from Harvard Business Review.
Hitchhiker's Guide To... is offline   Reply With Quote
Reply

  Irish Poker Boards > Community, Tech, and Feedback > Personal Logs

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



All times are GMT. The time now is 10:31.