Home

Welcome to www.davidbasalla3d.com

  • Ticket Checker

    I’ve made another thing (it’s been quite a while judging by the last post)! Over the weekend, I set up a script that periodically checks and emails me if tickets have become available on a certain website for the date that I’m interested in. It’s basically some ruby code that uses Watir to analyse a webpage, Mailgun to send me an email and is hosted on Heroku with the scheduler plugin.

    For anyone living in London, it’s an unavoidable fact that you have to book tickets for anything good way in advance. This has bitten me far too many times in the past. With my sister arriving in a month’s time and all bars and restaurants at the Sky Garden being booked out, I was finally motivated to do something about it. Sky Garden have a number of free tickets that they make available 3 weeks in advance. So rather than me having to click on the site every single day to check for tickets, I set out to find a more programmatic solution…

    First I had a look in the Chrome web console and the Tamper plug-in, which allows you to intercept HTTP requests. The ticket availability on the website is presented in a table, with the individual days being individual clickable cells. Clicking on an individual date showed requests to a specific booking related API. However curling the API directly returned permissions errors. I assumed that this had to do with some sort of fancy session set up and I didn’t have the time or dedication to figure out how that worked. Instead I realised that I would only need to query the table cells on the webpage and see if they were marked as available somehow. Indeed I was in luck, each available table cell had the available CSS class assigned to it. That meant I could use HTML parsing libraries such as Nokogiri to analyse the page and find the relevant elements.

    Next I checked the robots.txt file to see if there was any guidance to not crawl the site. In any case I figured that the traffic I would be generating would be minimal, as I would only have to check for availability once every hour or day or so. I didn’t find anything specific to dissuade me from my mission.

    I then started writing a little ruby script, using the Watir to simulate the browser. Originally I tried just using other popular crawling gems such as plain Nokogiri or Mechanize but realised that they could not sufficiently deal with the AJAX calls that the site was using. Instead Watir uses a proper browser of your choice which you can pass a wait time to. At first I went with the Chrome web driver. Later when deploying this to Heroku, I switched to Phantomjs since Heroku doesn’t support apps running as a graphical based app, and the headless Phantomjs browser worked just as well.

    The other bit to mention would be Mailgun. This is the first time I’ve dealt with sending emails from a service, and Mailgun was pleasantly simple to use, once I had found the right tutorials and realised you can send emails straight from the dummy domain that Mailgun gives you out of a box.

    Finally I deployed everything to Heroku, and use the schedule plugin to set appropriate intervals to run the script. It would then email me whenever tickets became available for a target date of my choosing. I was pleased to find that I could run everything on free accounts, so didn’t actually have to spend any money to host anything! The script could obviously much more clever and configurable, but it was a fun project for a few hours, and most importantly I got my free tickets to Sky Garden!!

    I’m aiming to put the code up on Github at some point, will post a link here! In the mean time, see the code below:

    TicketChecker class:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
      require 'watir'
      require 'pry'
    
      require_relative 'emailer'
      require_relative 'availability_parser'
    
      class TicketChecker
        TARGET_DATE = 4
        TARGET_MONTH = "June"
        BOOKING_URL = "https://skygarden_url_for_booking_tickets"
        BROWSER_WAIT_TIME = 5
    
        def call
          if checker.is_available?
            text = "TARGET DATE (#{TARGET_MONTH} #{TARGET_DATE}) IS AVAILABLE"
            emailer.send_email(text)
            puts text
          end
    
          browser.close
        end
    
      private
    
        def checker
          @_checker ||= AvailabilityParser.new(TARGET_DATE, TARGET_MONTH, browser)
        end
    
        def browser
          @_browser ||= begin
            web_browser = Watir::Browser.new :phantomjs
            web_browser.goto(BOOKING_URL)
            web_browser.driver.manage.timeouts.implicit_wait = BROWSER_WAIT_TIME
            web_browser
          end
        end
    
        def emailer
          @_emailer ||= Emailer.new
        end
      end
    
      TicketChecker.new.call
    

    AvailabilityParser class:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
      class AvailabilityParser
        AVAILABLE_KEYWORD = "available"
    
        attr_reader :target_day, :target_month, :browser
    
        def initialize(target_day, target_month, browser)
          @target_day = target_day
          @target_month = target_month
          @browser = browser
        end
    
        def is_available?
          available_tds_text.include?(target_day.to_s)
        end
    
      private
    
        def available_tds_text
          available_tds.map(&:text)
        end
    
        def available_tds
          tds.select { |c| c.class_name.include?(AVAILABLE_KEYWORD)}
        end
    
        def tds
          browser.iframes.last.tds
        end
      end
    

    Emailer class:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
      require 'mailgun' 
    
      class Emailer
        FROM_DOMAIN = "mail_gun_domain.com"
        TO = "myemail@gmail.com"
        SUBJECT = 'Skygarden Ticket Availability'
        API_KEY = "MAILGUN_API_KEY"
    
        def initialize
        end
    
        def send_email(text)
          client.send_message(FROM_DOMAIN, sender_params.merge(text: text))
        end
    
      private
    
        def client
          @_client ||= Mailgun::Client.new(API_KEY)
        end
    
        def sender_params
          { 
            from: "mailgun@#{FROM_DOMAIN}",
            to: TO, 
            subject: SUBJECT,
          }
        end
      end
    

  • Match Three Game

    So I’ve had a go at making a Match Three style game, similar to Bejeweled and Candy Crush. Check out the work in progress here! I figured it would be a good chance to play around with JavaScript and the Canvas element a bit more. In the end I used RequireJS for managing dependencies, FabrisJS for drawing on the canvas and Underscore for its more convenient array methods. Everything else is just vanilla JS.

    One major benefit of this little toy project is that I finally used JavaScript promises! When initially implementing the animation of the gems, I used callbacks from when an animation was complete to start the next ‘wave’ of animations. That quickly got so complex that it was really hard to reason about the board state. Also I had to somehow manage simultaneous animation of multiple gems, and wait until ALL of them had finished. After a brief pub chat of me relaying my troubles to friends and colleagues, I was told to check out promises! And they’re amazing!! They solved all my problems pretty quickly, especially using Promise.all for running simultaneous asynchronous operations and signalling when all of them have finished.

    Another thing that helped with development was that I added a little map loader which allowed me to define test cases (e.g. of just one row with one possible match). This helped immensely with checking that the matching logic and animation behaviour worked as intended. Ideally I would be able to write some sort of tests, but for the moment these little test maps work pretty well.

  • Isometric Game Test

    Over the weekend I managed to take some WebGL libraries (ThreeJS, BabylonJS) for a spin, and made a little demo reminiscent of isometric RPG-style games. Sadly you can only walk around with the WASD keys at the moment, and I still have to implement proper movement and any sort of collision detection. But playing around with the libraries was fun! I started with ThreeJS but ran into problems with its shadowing fairly quickly (shadows turn everything completely black even with other lights present). Also the public Github repo has loads of unmerged open PR’s, so I looked for alternatives. I came across BabylonJS which I’ve liked a lot, you can see the results in the demo.

    The map is defined as square of symbols in text format, where the different types of objects are identified by letter (P for Pillar) and a number for rotation (1 for 90 degrees, 2 for 180, etc). It uses inbuilt primitives such as Cubes, Cylinders as well as custom modeled objects in OBJ format. I’ve also downloaded the model and walkcycle from the BabylonJS playground website.

    I’ve had some issues with loading custom OBJ files, especially handling the asynchronous loading. Also the fact that you can only have a maximum number of 4 lights active is a bit sobering, but I suppose I shouldn’t expect full blown rendering capabilities which I’m used to from Maya, Renderman, etc. Anyway, it’s been a lot of fun, will see how much further I get with it.

  • CodeBar

    I went to my first CodeBar session! codebar.io is a great non-for-profit organisation that aims to teach “underrepresented groups to learn programming in a safe and collaborative environment”. The events are run as weekly workshops where tutors meet with students to work on a set of tutorials or their own projects. This event was organised by Tatiana Soukiassian who works as a developer at GDS. I initially only volunteered for setting up the venue and showing people to the right floor, but by the end it turned out that two people still needed a tutor for an introduction to HTML. So I spent the next 2 hours helping out by going through the tutorials with them. We even branched out a bit and had a quick look at using Bootstrap. I found it really rewarding and enjoyable, would definitely recommend for anyone to get involved!

  • Property Prices UK

    I wrote a Rails app to display property prices in the UK over the last 10 years (2006-2015), based on data from the Land Registry. Written with Rails, BootStrap, ChartJS, hosted on Heroku. Inspired by a coding test at a start up I took a while ago. Planning to add a paginated listing of the individual sales as well as a breakdown by type of property.

    LINK


  • For more ramblings, check out the Posts section.