ENH: service account credentials support + optional verbosity

closes #11881
closes #8489

Author: chromy

doc/source/io.rst

@@ -4093,6 +4093,34 @@ The key functions are:
 
 .. _io.bigquery_reader:
 
+
+Authentication
+''''''''''''''
+
+Authentication is possible with either user account credentials or service account credentials.
+
+Authenticating with user account credentials is as simple as following the prompts in a browser window
+which will be automatically opened for you. You will be authenticated to the specified
+``BigQuery`` account via Google's ``Oauth2`` mechanism. Additional information on the
+authentication mechanism can be found `here <https://developers.google.com/identity/protocols/OAuth2#clientside/>`__.
+
+Authentication with service account credentials is possible via the `'private_key'` parameter. This method
+is particularly useful when working on remote servers (eg. jupyter iPython notebook on remote host).
+The remote authentication using user account credentials is not currently supported in Pandas.
+Additional information on service accounts can be found
+`here <https://developers.google.com/identity/protocols/OAuth2#serviceaccount>`__.
+
+.. note::
+
+   The `'private_key'` parameter can be set to either the file path of the service account key
+   in JSON format, or key contents of the service account key in JSON format.
+
+.. note::
+
+   A private key can be obtained from the Google developers console by clicking
+   `here <https://console.developers.google.com/permissions/serviceaccounts>`__. Use JSON key type.
+
+
 Querying
 ''''''''
 
@@ -4107,12 +4135,6 @@ into a DataFrame using the :func:`~pandas.io.gbq.read_gbq` function.
 
    data_frame = pd.read_gbq('SELECT * FROM test_dataset.test_table', projectid)
 
-You will then be authenticated to the specified BigQuery account
-via Google's Oauth2 mechanism. In general, this is as simple as following the
-prompts in a browser window which will be opened for you. Should the browser not
-be available, or fail to launch, a code will be provided to complete the process
-manually.  Additional information on the authentication mechanism can be found
-`here <https://developers.google.com/accounts/docs/OAuth2#clientside/>`__.
 
 You can define which column from BigQuery to use as an index in the
 destination DataFrame as well as a preferred column order as follows:
@@ -4125,7 +4147,7 @@ destination DataFrame as well as a preferred column order as follows:
 
 .. note::
 
-   You can find your project id in the `BigQuery management console <https://code.google.com/apis/console/b/0/?noredirect>`__.
+   You can find your project id in the `Google developers console <https://console.developers.google.com>`__.
 
 
 .. note::
@@ -4134,6 +4156,7 @@ destination DataFrame as well as a preferred column order as follows:
 
 .. _io.bigquery_writer:
 
+
 Writing DataFrames
 ''''''''''''''''''
 

doc/source/whatsnew/v0.18.0.txt

@@ -201,6 +201,7 @@ Other enhancements
 - ``DataFrame.quantile`` and ``Series.quantile`` now accept ``interpolation`` keyword (:issue:`10174`).
 - ``DataFrame.select_dtypes`` now allows the ``np.float16`` typecode (:issue:`11990`)
 - ``pivot_table()`` now accepts most iterables for the ``values`` parameter (:issue:`12017`)
+- Added Google ``BigQuery`` service account authentication support, which enables authentication on remote servers. (:issue:`11881`). For further details see :ref:`here <io.bigquery_authentication>`
 
 .. _whatsnew_0180.api_breaking:
 

pandas/core/frame.py

@@ -859,7 +859,7 @@ def to_dict(self, orient='dict'):
             raise ValueError("orient '%s' not understood" % orient)
 
     def to_gbq(self, destination_table, project_id, chunksize=10000,
-               verbose=True, reauth=False, if_exists='fail'):
+               verbose=True, reauth=False, if_exists='fail', private_key=None):
         """Write a DataFrame to a Google BigQuery table.
 
         THIS IS AN EXPERIMENTAL LIBRARY
@@ -883,14 +883,18 @@ def to_gbq(self, destination_table, project_id, chunksize=10000,
             'fail': If table exists, do nothing.
             'replace': If table exists, drop it, recreate it, and insert data.
             'append': If table exists, insert data. Create if does not exist.
+        private_key : str (optional)
+            Service account private key in JSON format. Can be file path
+            or string contents. This is useful for remote server
+            authentication (eg. jupyter iPython notebook on remote host)
 
             .. versionadded:: 0.17.0
         """
 
         from pandas.io import gbq
         return gbq.to_gbq(self, destination_table, project_id=project_id,
                           chunksize=chunksize, verbose=verbose, reauth=reauth,
-                          if_exists=if_exists)
+                          if_exists=if_exists, private_key=private_key)
 
     @classmethod
     def from_records(cls, data, index=None, exclude=None, columns=None,