Google BigQuery ODBC driver

We've released an ODBC driver for Google BigQuery. The driver lets you query and update BigQuery data in applications other than the Google BigQuery console. We thought we'd share some of the things you can do in these third-party applications (for example, Perl, PHP, Excel, and Oracle).

Installing the BigQuery ODBC driver

1. Install and license the Google BigQuery ODBC driver on the machine where the client application is installed.

   For example, if you want to work with BigQuery data in Excel, install the BigQuery ODBC driver on the machine where Excel is installed.

   To do this, execute the file distribution, and follow the onscreen instructions.

   The install program starts the Easysoft License Manager, because you cannot use your Easysoft product until a license is obtained.

   The following types of license are available:

   • Free time-limited trial license, which gives you free and unrestricted use of the product for a limited period (usually 14 days).
   • Full license if you have purchased the product. On purchasing the product, you are given an authorization code, which you use to obtain a license.
2. In Easysoft License Manager, enter your details.

   Complete the Name, E-Mail Address, and Company fields.

   The E-Mail Address must be the same as the address used to register and download from the Easysoft web site or you will be unable to obtain trial licenses.

3. Choose Request License.

   You're asked for a license type.

4. Do one of the following:
   • For a trial license, choose Time Limited Trial and then choose Next.

     The License Manager asks what software you are licensing. Select your product from the drop-down list and then choose Next.

   • If you have obtained an authorization code for a purchased license, select Non-expiring License and then choose Next.

     The License Manager requests your authorization code.

     Enter the authorization code and then choose Next.

5. The License Manager displays a summary of the information you entered and allows you to choose the method of applying for your license.
6. Do one of the following:
   • Choose On-line Request if your machine is connected to the internet and can make outgoing connections to port 8884.

     The License Manager then sends a request to the Easysoft license server to activate your license key automatically. This is the quickest method and results in your details being entered immediately into our support database.

   • The remaining three options (Email Request, Print Request, and View Request) are all ways to obtain a license if your machine is off-line (does not have a connection to the internet).

     Each of these methods involves providing Easysoft with information including your machine number (a number unique to your machine) and then waiting to receive your license key.

     Instead of emailing your details to Easysoft, you can enter them directly at the Easysoft web site and your license key will be emailed to you automatically.

     To use this method, choose View Request, and then visit:

     • Trial license page
     • Full license page

     In the licensing page, enter your machine number (and authorization code for a purchased license), choose Submit and your license key will be emailed to you.

     When you receive the license key, you can activate it either by double-clicking the email attachment or by choosing Enter License on the License Manager main screen and pasting the license key into the dialog box.

7. Choose Finish to exit the License Manager.

   The installation is complete.

Configuring an ODBC data source

Before the BigQuery ODBC driver can be used to connect an application to BigQuery, it's necessary to configure an ODBC data source. An ODBC data source stores the connection details for the target database (in this case, BigQuery) and the ODBC driver that is required to connect to it (in this case, the BigQuery ODBC driver).

ODBC data sources are configured in ODBC Data Source Administrator, which is included with Windows. How ODBC Data Source Administrator is run depends on whether your application is 32-bit or 64-bit:

• My application is 32-bit

  In the Windows Run dialog box, enter:

  %windir%\syswow64\odbcad32.exe

• My application is 64-bit

  Open System and Security > Administrative Tools in Control Panel, and then open ODBC Data Sources (64-bit). (On Windows Server 2003 and earlier, the Control Panel applet that launches ODBC Data Source Administrator is labelled Data Sources.)

The following table shows the architecture of some popular applications.

select SERVERPROPERTY('edition')

In ODBC Data Source Administrator:

1. Choose the System DSN tab, and then choose Add.
2. In the Create New Data Source dialog box, choose Easysoft ODBC-BigQuery Driver, and then choose Finish.
3. Complete these fields in the Easysoft ODBC-BigQuery Driver DSN Setup dialog box:

   Setting	Value
   DSN	BigQuery
   Refresh Token	Choose the Authenticate and obtain token and grant the BigQuery ODBC driver permission to access your BigQuery data to populate this field.

4. In your application, connect to your newly configured data source and run a sample query. For example:

   select * from MyTable limit 10

Work with BigQuery data in Perl

Strawberry Perl is a Perl distribution for Windows that includes the necessary middleware layers (Perl DBI and Perl DBD::ODBC) to enable the BigQuery ODBC driver to connect your Perl applications to BigQuery.

1. Here's a Perl script that retrieves some BiqQuery data:

   #!/usr/bin/perl -w
   use strict;
   use DBI;
   my $dbh = DBI-> connect('dbi:ODBC:MyBigQueryDataSource');
   
   my $sql = "SELECT residential_ratio  FROM `bigquery-public-data.hud_zipcode_crosswalk.census_tracts_to_zipcode` LIMIT 10";
   
   # Prepare the statement.
   my $sth = $dbh->prepare($sql)
       or die "Can't prepare statement: $DBI::errstr";
   
   # Execute the statement.
   $sth->execute();
   
   my($BigQueryCol);
   
   # Fetch and display the result set value.
   while(($BigQueryCol) = $sth->fetchrow()){
      print("$BigQueryCol\n");                   
   }
   
   $dbh->disconnect if ($dbh);

2. Here's a Perl script that inserts some BigQuery data:

   #!/usr/bin/perl -w
   use strict;
   use warnings;
   use DBI;
   
   my $data_source = q/dbi:ODBC:MyBigQueryDataSource/;
   
   my $h = DBI-"connect($data_source)
       or die "Can't connect to $data_source: $DBI::errstr";
   $h->{RaiseError} = 1;
   
   my $s = $h->prepare(q/insert into MyTable(MyStringCol, MyIntegerCol, MyFloatCol, MyNumericCol, MyBooleanCol,MyDateCol,MyTimeCol,MyDateTimeCol,MyTimeStampCol) values(?,?,?,?,?,?,?,?,?)/);
   my $string='MyValue';
   my $int=46683;
   my $float=8.3;
   my $numeric=0.00333;
   my $bool=0;
   my $date='2021-01-23';
   my $time='00:00:20';
   my $datetime='2021-02-05 00:00:00';
   my $timestamp='2019-01-27 00:00:00';
   
   $s->bind_param(1, $string, DBI::SQL_WVARCHAR);
   $s->bind_param(2, $int, DBI::SQL_INTEGER);
   $s->bind_param(3, $float, DBI::SQL_FLOAT);
   $s->bind_param(4, $numeric, DBI::SQL_NUMERIC);
   #$s->bind_param(5, $bignumeric, DBI::SQL_WVARCHAR);
   $s->bind_param(5, $bool, DBI::SQL_TINYINT);
   $s->bind_param(6, $date, DBI::SQL_DATE);
   $s->bind_param(7, $time, DBI::SQL_TIME);
   $s->bind_param(8, $datetime, DBI::SQL_DATETIME);
   $s->bind_param(9, $timestamp, DBI::SQL_TIMESTAMP);
   $s->execute($string,$int,$float,$numeric,$bool,$date,$time,$datetime,$timestamp);
   $h->disconnect;
   	  

3. Here's a Perl script that updates a BigQuery record:

   #!/usr/bin/perl -w
   use strict;
   use warnings;
   use DBI;
   
   my $data_source = q/dbi:ODBC:MyBigQueryDataSource/;
   
   my $h = DBI->connect($data_source)
       or die "Can't connect to $data_source: $DBI::errstr";
   $h->{RaiseError} = 1;
   
   my $s = $h->prepare(q/UPDATE MyTestTable SET MyStringCol = ? WHERE MyStringCol = 'MyValue' /);
   my $string='MyNewValue';
   
   $s->bind_param(1, $string, DBI::SQL_WVARCHAR);
   
   $s->execute($string);
   $h->disconnect;

Work with BigQuery data in PHP

1. Here's a PHP script that retrieves some BigQuery data:

   <?php
     $con = odbc_connect("MyBigQueryDataSource", "", "");
     $err = odbc_errormsg();
     if (strlen($err) <> 0) {
       echo odbc_errormsg();
     } else {
       $rs2 = odbc_exec($con, "select MyCol from MyTable");
       odbc_result_all($rs2);
       odbc_close($con);
     }
   ?>

Connecting Excel to BigQuery

Follow these steps to return data from BigQuery to Microsoft Excel by using Microsoft Query:

1. On the Data tab, choose New Query > From Other Source > From ODBC.
2. Choose the BigQuery ODBC data source when prompted.
3. Choose a table from the available data sets.
4. Choose the Load to return the BiqQuery data to the worksheet.

   Note that for large result sets, you may have to have to filter the data using Excel before the data can be returned to the worksheet.

Connect to BigQuery from Oracle

1. Create a DG4ODBC init file on your Oracle machine. To do this, change to the %ORACLE_HOME%\hs\admin directory. Create a copy of the file initdg4odbc.ora. Name the new file initbq.ora.

   Note In these instructions, replace %ORACLE_HOME% with the location of your Oracle HOME directory. For example, C:\oraclexe\app\oracle\product\11.2.0\server.

2. Ensure these parameters and values are present in your init file:

   HS_FDS_CONNECT_INFO = MyBigQueryDataSource

3. Comment out the line that enables DG4ODBC tracing. For example:

   #HS_FDS_TRACE_LEVEL = <trace_level>

4. Add an entry to %ORACLE_HOME%\network\admin\listener.ora that creates a SID_NAME for DG4ODBC. For example:

   SID_LIST_LISTENER =
    (SID_LIST =
      (SID_DESC=
        (SID_NAME=bq)
        (ORACLE_HOME=%ORACLE_HOME%)
        (PROGRAM=dg4odbc)
      )
    )

5. Add a DG4ODBC entry to %ORACLE_HOME%\network\admin\tnsnames.ora that specifies the SID_NAME created in the previous step. For example:

   BQ =
     (DESCRIPTION =
       (ADDRESS = (PROTOCOL = TCP)(HOST = oracle_host)(PORT = 1521))
       (CONNECT_DATA =
         (SID = bq)
       )
       (HS = OK)
     )

   Replace oracle_host with the host name of your Oracle machine.

6. Start (or restart) the Oracle Listener:

   cd %ORACLE_HOME%\bin
   lsnrctl stop
   lsnrctl start

7. Connect to your Oracle database in SQL*Plus.
8. In SQL*Plus, create a database link for the target BigQuery instance. For example:

   CREATE PUBLIC DATABASE LINK bqlink CONNECT TO "mydummyuser" IDENTIFIED BY "mydummypassword" USING 'bq';

9. Try querying your BigQuery data. For example:

   SELECT * FROM "MyTable"@BQLINK;

Notes

• If you have problems connecting to BigQuery from Oracle, enable DG4ODBC tracing and check the trace files written to the %ORACLE_HOME%\hs\trace directory. To enable DG4ODBC tracing, add the line HS_FDS_TRACE_LEVEL = DEBUG to initbq.ora and then start or restart the Oracle listener. If the trace directory does not exist, create it.
• If you enable ODBC Driver Manager tracing, but do not get a trace file or get an empty trace file (check for C:\SQL.log), change the trace file location to the Windows TEMP directory. For example, C:\Windows\Temp\SQL.log.