User Tools

Site Tools


Fetching a secure web page

by Richard Russell, July 2009, updated July 2018

The procedure listed below fetches the contents of a secure (https:) web page to a specified file; it uses the OpenSSL library and requires the files libssl-1_1.dll and libcrypto-1_1.dll to be available. They can be obtained from this page; download and install Win32OpenSSL_Light-1_1_0h.exe from there. You should normally choose the option of installing the DLLs in the Windows system directory.

If you want to distribute libssl-1_1.dll and libcrypto-1_1.dll with your application then store them in @dir$ and embed them in the executable. Alternatively you can put them in @lib$ (or a sub-directory) but in that case you will need to amend the procedure below to load them explicitly from that location.

The procedure should be called in the following context:

      port$ = "443" : REM https
      host$ = ""
      page$ = "/sslcheck.html"
      file$ = @tmp$ + "sslcheck.html"
      PROCsslfetch(port$, host$, page$, file$)

This fetches the page to the file sslcheck.html.

Here is the procedure:

      DEF PROCsslfetch(port$, host$, page$, file$)
      LOCAL libssl%, libeay%, meth%, ctx%, sock%, temp%, res%, ssl%, sbio%, file%
      LOCAL req$, buf&()
      FIONBIO = &8004667E
      BIO_NOCLOSE = 0
      BUFSIZ = 256
      SYS "LoadLibrary", "libcrypto-1_1.dll" TO libeay%
      IF libeay% = 0 PROCsslcleanup : ERROR 100, "Cannot load libcrypto-1_1.dll"
      SYS "GetProcAddress", libeay%, "BIO_new_socket"   TO `BIO_new_socket`
      SYS "LoadLibrary", "libssl-1_1.dll" TO libssl%
      IF libssl% = 0 PROCsslcleanup : ERROR 100, "Cannot load libssl-1_1.dll"
      SYS "GetProcAddress", libssl%, "TLSv1_2_method"   TO `TLSv1_2_method`
      SYS "GetProcAddress", libssl%, "SSL_CTX_new"      TO `SSL_CTX_new`
      SYS "GetProcAddress", libssl%, "SSL_new"          TO `SSL_new`
      SYS "GetProcAddress", libssl%, "SSL_set_bio"      TO `SSL_set_bio`
      SYS "GetProcAddress", libssl%, "SSL_connect"      TO `SSL_connect`
      SYS "GetProcAddress", libssl%, "SSL_write"        TO `SSL_write`
      SYS "GetProcAddress", libssl%, "SSL_read"         TO `SSL_read`
      SYS "GetProcAddress", libssl%, "SSL_CTX_free"     TO `SSL_CTX_free`
      REM Create SSL context:
      SYS `TLSv1_2_method` TO meth%
      SYS `SSL_CTX_new`, meth% TO ctx%
      IF ctx% = 0 PROCsslcleanup : ERROR 100, "SSL_CTX_new failed"
      REM Connect the TCP socket:
      sock% = FN_tcpconnect(host$, port$)
      IF sock% < 0 PROCsslcleanup : ERROR 100, "Cannot connect to " + host$
      temp% = 0
      SYS `ioctlsocket`, sock%, FIONBIO, ^temp% TO res%
      IF res% PROCsslcleanup : ERROR 105, "Cannot set socket to blocking"
      REM Connect the SSL socket:
      SYS `SSL_new`, ctx% TO ssl%
      SYS `BIO_new_socket`, sock%, BIO_NOCLOSE TO sbio%
      SYS `SSL_set_bio`, ssl%, sbio%, sbio%
      SYS `SSL_connect`, ssl% TO res%
      IF res% <= 0 PROCsslcleanup : ERROR 100, "SSL connect failed: " + STR$res%
      REM Request the page:
      req$  = "GET " + page$ + " HTTP/1.0" + CHR$13 + CHR$10
      req$ += "User-Agent: BB4W" + CHR$13 + CHR$10
      req$ += "Host: " + host$ + ":" + port$ + CHR$13 + CHR$10
      req$ += CHR$13 + CHR$10
      SYS `SSL_write`, ssl%, req$, LEN(req$) TO res%
      IF res% <> LEN(req$) PROCsslcleanup : ERROR 100, "SSL write failed: " + STR$res%
      REM Copy the requested page to a file:
      DIM buf&(BUFSIZ-1)
      file% = OPENOUT(file$)
        SYS `SSL_read`, ssl%, ^buf&(0), BUFSIZ TO res%
        IF res% > 0 SYS "WriteFile", @hfile%(file%), ^buf&(0), res%, ^temp%, 0
      UNTIL res% <= 0
      CLOSE #file%
      IF res% PROCsslcleanup : ERROR 100, "SSL read failed: " + STR$res%
      REM Tidy up before exit:
      DEF PROCsslcleanup
      sock% += 0 : IF sock% PROC_closesocket(sock%) : sock% = 0
      ctx% += 0 :  IF ctx% SYS `SSL_CTX_free`, ctx% : ctx% = 0
      libssl% += 0 : IF libssl% SYS "FreeLibrary", libssl% : libssl% = 0
      libeay% += 0 : IF libeay% SYS "FreeLibrary", libeay% : libeay% = 0
This website uses cookies. By using the website, you agree with storing cookies on your computer. Also you acknowledge that you have read and understand our Privacy Policy. If you do not agree leave the website.More information about cookies
fetching_20a_20secure_20web_20page.txt · Last modified: 2018/07/30 23:42 by richardrussell