Writing Tables¶
Use the pandas_gbq.to_gbq() function to write a
pandas.DataFrame object to a BigQuery table.
import pandas
import pandas_gbq
# TODO: Set project_id to your Google Cloud Platform project ID.
# project_id = "my-project"
# TODO: Set table_id to the full destination table ID (including the
# dataset ID).
# table_id = 'my_dataset.my_table'
df = pandas.DataFrame(
{
"my_string": ["a", "b", "c"],
"my_int64": [1, 2, 3],
"my_float64": [4.0, 5.0, 6.0],
"my_bool1": [True, False, True],
"my_bool2": [False, True, False],
"my_dates": pandas.date_range("now", periods=3),
}
)
pandas_gbq.to_gbq(df, table_id, project_id=project_id)
The destination table and destination dataset will automatically be created if they do not already exist.
Writing to an Existing Table¶
Use the if_exists argument to dictate whether to 'fail',
'replace' or 'append' if the destination table already exists. The
default value is 'fail'.
For example, assume that if_exists is set to 'fail'. The following snippet will raise
a TableCreationError if the destination table already exists.
import pandas_gbq
pandas_gbq.to_gbq(
df, 'my_dataset.my_table', project_id=projectid, if_exists='fail',
)
If the if_exists argument is set to 'append', the destination
dataframe will be written to the table using the defined table schema and
column types. The dataframe must contain fields (matching name and type)
currently in the destination table.
Inferring the Table Schema¶
The to_gbq() method infers the BigQuery table schema based
on the dtypes of the uploaded DataFrame.
dtype |
BigQuery Data Type |
|---|---|
i (integer) |
INTEGER |
b (boolean) |
BOOLEAN |
f (float) |
FLOAT |
O (object) |
STRING |
S (zero-terminated bytes) |
STRING |
U (Unicode string) |
STRING |
M (datetime) |
TIMESTAMP |
If the data type inference does not suit your needs, supply a BigQuery schema
as the table_schema parameter of to_gbq().
Troubleshooting Errors¶
If an error occurs while writing data to BigQuery, see Troubleshooting BigQuery Errors.
