Getting Started with Bigtable Table Administrative Operations

This example illustrates how to use the BigtableTableAdminClient class to:

• Create a table.
• List tables.
• Retrieve table metadata.
• Modify existing tables.
• Delete all the rows in a table.
• Delete a table.

Running the example

This example uses the Cloud Bigtable C++ Client Library to communicate with Cloud Bigtable.

To run the example program, follow the instructions for the example on GitHub.

Include the Necessary Headers

The example uses the following headers:

#include "google/cloud/bigtable/admin/bigtable_table_admin_client.h"
#include "google/cloud/bigtable/resource_names.h"

Define aliases

To make the example less verbose we define some aliases:

  namespace cbt = ::google::cloud::bigtable;
  namespace cbta = ::google::cloud::bigtable_admin;
  using ::google::cloud::StatusOr;

Connect to the Cloud Bigtable Table Admin Endpoint.

Use the bigtable_admin::BigtableTableAdminClient class to obtain information about Cloud Bigtable tables and to modify them:

  cbta::BigtableTableAdminClient admin(
      cbta::MakeBigtableTableAdminConnection());

See also

https://cloud.google.com/bigtable/docs/overview for an overview of the Cloud Bigtable storage model, including an introduction to important Cloud Bigtable concepts, such as row keys, column families, columns, and cells.

Creating a Table

Use BigtableTableAdminClient::CreateTable() to create a new table:

  // Define the schema
  auto constexpr kSecondsPerDay =
      std::chrono::seconds(std::chrono::hours(24)).count();
 
  google::bigtable::admin::v2::Table t;
  auto& families = *t.mutable_column_families();
  families["fam"].mutable_gc_rule()->set_max_num_versions(10);
  families["foo"].mutable_gc_rule()->mutable_max_age()->set_seconds(
      3 * kSecondsPerDay);
 
  std::cout << "Creating a table:\n";
  std::string instance_name = cbt::InstanceName(project_id, instance_id);
  StatusOr<google::bigtable::admin::v2::Table> schema =
      admin.CreateTable(instance_name, table_id, std::move(t));
  std::cout << "DONE\n";

Note that when you create a table you can define one or more column families for the table, and each column families is configured with a garbage collection rule to automatically delete excess cells.

See also

Configure garbage collection to learn about the garbage collection rules.

Listing Tables in an Instance

Get all the tables in an instance using BigtableTableAdminClient::ListTables():

  std::cout << "Listing tables:\n";
  google::bigtable::admin::v2::ListTablesRequest list_req;
  list_req.set_parent(instance_name);
  list_req.set_view(google::bigtable::admin::v2::Table::NAME_ONLY);
  for (auto& table : admin.ListTables(std::move(list_req))) {
    if (!table) throw std::move(table).status();
    std::cout << "    " << table->name() << "\n";
  }
  std::cout << "DONE\n";

Note how this example uses a TableView parameter to query only a subset of the table metadata.

Retrieve Table Metadata

Or retrieve the metadata, including column family definitions, using BigtableTableAdminClient::GetTable():

  std::cout << "Get table metadata:\n";
  google::bigtable::admin::v2::GetTableRequest get_req;
  get_req.set_name(schema->name());
  get_req.set_view(google::bigtable::admin::v2::Table::FULL);
  StatusOr<google::bigtable::admin::v2::Table> table =
      admin.GetTable(std::move(get_req));
  if (!table) throw std::move(table).status();
  std::cout << "Table name : " << table->name() << "\n";
 
  std::cout << "List table families and GC rules:\n";
  for (auto const& kv : table->column_families()) {
    std::string const& family_name = kv.first;
    google::bigtable::admin::v2::ColumnFamily const& metadata = kv.second;
    std::cout << "Column Family :" << family_name << "\t"
              << metadata.DebugString() << "\n";
  }
  std::cout << "DONE\n";

See also

Configure garbage collection to learn about the garbage collection rules.

Modify a Column Family

You can also add new column families, delete a column family, or update an existing column family:

  std::cout << "Update a column family GC rule:\n";
  using ::google::bigtable::admin::v2::ModifyColumnFamiliesRequest;
  ModifyColumnFamiliesRequest::Modification mod;
  mod.set_id("fam");
  mod.mutable_update()->mutable_gc_rule()->set_max_num_versions(5);
 
  StatusOr<google::bigtable::admin::v2::Table> updated_schema =
      admin.ModifyColumnFamilies(table->name(), {std::move(mod)});
  if (!updated_schema) {
    throw std::move(updated_schema).status();
  }
  std::cout << "Schema modified to: " << updated_schema->DebugString() << "\n";

Warning

Note that deleting a column family also deletes all the data in the column family. Cloud Bigtable will delete an existing column family even if you have data in it. For mission critical data you must ensure that the data is backed up dropping any column families.

Delete all the Rows in a Table

Use BigtableTableAdminClient::DropRowRange() to delete all the rows in a table:

  std::cout << "Deleting all the rows in " << table_id << "\n";
  google::bigtable::admin::v2::DropRowRangeRequest drop_req;
  drop_req.set_name(table->name());
  drop_req.set_delete_all_data_from_table(true);
  google::cloud::Status status = admin.DropRowRange(std::move(drop_req));
  if (!status.ok()) throw std::runtime_error(status.message());
  std::cout << "DONE\n";

Delete a Table

Finally use BigtableTableAdminClient::DeleteTable() to delete a a table:

  std::cout << "Deleting table:\n";
  google::cloud::Status delete_status = admin.DeleteTable(table->name());
  if (!delete_status.ok()) throw std::runtime_error(delete_status.message());
  std::cout << "DONE\n";

Warning

Note that this function will delete the table even if it contains data, for mission critical data you must ensure that the data is backed up before calling this function.

Put it all together

Here is the full example

 
#include "google/cloud/bigtable/admin/bigtable_table_admin_client.h"
#include "google/cloud/bigtable/resource_names.h"
#include "google/cloud/bigtable/examples/bigtable_examples_common.h"
#include "google/cloud/bigtable/testing/cleanup_stale_resources.h"
#include "google/cloud/bigtable/testing/random_names.h"
#include "google/cloud/internal/getenv.h"
#include "google/cloud/internal/random.h"
#include "google/cloud/log.h"
#include <chrono>
#include <iostream>
 
namespace {
 
using ::google::cloud::bigtable::examples::Usage;
 
void HelloWorldTableAdmin(std::vector<std::string> const& argv) {
  if (argv.size() != 3) {
    throw Usage{
        "hello-world-table-admin <project-id> <instance-id> "
        "<table-id>"};
  }
  std::string const& project_id = argv[0];
  std::string const& instance_id = argv[1];
  std::string const& table_id = argv[2];
 
  namespace cbt = ::google::cloud::bigtable;
  namespace cbta = ::google::cloud::bigtable_admin;
  using ::google::cloud::StatusOr;
 
  // Connect to the Cloud Bigtable admin endpoint.
  cbta::BigtableTableAdminClient admin(
      cbta::MakeBigtableTableAdminConnection());
 
  // Define the schema
  auto constexpr kSecondsPerDay =
      std::chrono::seconds(std::chrono::hours(24)).count();
 
  google::bigtable::admin::v2::Table t;
  auto& families = *t.mutable_column_families();
  families["fam"].mutable_gc_rule()->set_max_num_versions(10);
  families["foo"].mutable_gc_rule()->mutable_max_age()->set_seconds(
      3 * kSecondsPerDay);
 
  std::cout << "Creating a table:\n";
  std::string instance_name = cbt::InstanceName(project_id, instance_id);
  StatusOr<google::bigtable::admin::v2::Table> schema =
      admin.CreateTable(instance_name, table_id, std::move(t));
  std::cout << "DONE\n";
 
  std::cout << "Listing tables:\n";
  google::bigtable::admin::v2::ListTablesRequest list_req;
  list_req.set_parent(instance_name);
  list_req.set_view(google::bigtable::admin::v2::Table::NAME_ONLY);
  for (auto& table : admin.ListTables(std::move(list_req))) {
    if (!table) throw std::move(table).status();
    std::cout << "    " << table->name() << "\n";
  }
  std::cout << "DONE\n";
 
  std::cout << "Get table metadata:\n";
  google::bigtable::admin::v2::GetTableRequest get_req;
  get_req.set_name(schema->name());
  get_req.set_view(google::bigtable::admin::v2::Table::FULL);
  StatusOr<google::bigtable::admin::v2::Table> table =
      admin.GetTable(std::move(get_req));
  if (!table) throw std::move(table).status();
  std::cout << "Table name : " << table->name() << "\n";
 
  std::cout << "List table families and GC rules:\n";
  for (auto const& kv : table->column_families()) {
    std::string const& family_name = kv.first;
    google::bigtable::admin::v2::ColumnFamily const& metadata = kv.second;
    std::cout << "Column Family :" << family_name << "\t"
              << metadata.DebugString() << "\n";
  }
  std::cout << "DONE\n";
 
  std::cout << "Update a column family GC rule:\n";
  using ::google::bigtable::admin::v2::ModifyColumnFamiliesRequest;
  ModifyColumnFamiliesRequest::Modification mod;
  mod.set_id("fam");
  mod.mutable_update()->mutable_gc_rule()->set_max_num_versions(5);
 
  StatusOr<google::bigtable::admin::v2::Table> updated_schema =
      admin.ModifyColumnFamilies(table->name(), {std::move(mod)});
  if (!updated_schema) {
    throw std::move(updated_schema).status();
  }
  std::cout << "Schema modified to: " << updated_schema->DebugString() << "\n";
 
  std::cout << "Deleting all the rows in " << table_id << "\n";
  google::bigtable::admin::v2::DropRowRangeRequest drop_req;
  drop_req.set_name(table->name());
  drop_req.set_delete_all_data_from_table(true);
  google::cloud::Status status = admin.DropRowRange(std::move(drop_req));
  if (!status.ok()) throw std::runtime_error(status.message());
  std::cout << "DONE\n";
 
  std::cout << "Deleting table:\n";
  google::cloud::Status delete_status = admin.DeleteTable(table->name());
  if (!delete_status.ok()) throw std::runtime_error(delete_status.message());
  std::cout << "DONE\n";
}
 
void RunAll(std::vector<std::string> const& argv) {
  namespace examples = ::google::cloud::bigtable::examples;
  namespace cbt = ::google::cloud::bigtable;
  namespace cbta = ::google::cloud::bigtable_admin;
 
  if (!argv.empty()) throw examples::Usage{"auto"};
  if (!examples::RunAdminIntegrationTests()) return;
  examples::CheckEnvironmentVariablesAreSet({
      "GOOGLE_CLOUD_PROJECT",
      "GOOGLE_CLOUD_CPP_BIGTABLE_TEST_INSTANCE_ID",
  });
  auto const project_id =
      google::cloud::internal::GetEnv("GOOGLE_CLOUD_PROJECT").value();
  auto const instance_id = google::cloud::internal::GetEnv(
                               "GOOGLE_CLOUD_CPP_BIGTABLE_TEST_INSTANCE_ID")
                               .value();
 
  auto conn = cbta::MakeBigtableTableAdminConnection();
  cbt::testing::CleanupStaleTables(conn, project_id, instance_id);
 
  auto generator = google::cloud::internal::DefaultPRNG(std::random_device{}());
  auto table_id = cbt::testing::RandomTableId(generator);
 
  std::cout << "\nRunning the HelloWorldTableAdmin() example" << std::endl;
  HelloWorldTableAdmin({project_id, instance_id, table_id});
}
 
}  // namespace
 
int main(int argc, char* argv[]) try {
  google::cloud::bigtable::examples::Example example({
      {"auto", RunAll},
      {"hello-world-table-admin", HelloWorldTableAdmin},
  });
  return example.Run(argc, argv);
} catch (std::exception const& ex) {
  std::cerr << ex.what() << "\n";
  ::google::cloud::LogSink::Instance().Flush();
  return 1;
}