Infer Hive table schema automatically using Impala and Parquet

Опубликовано: 14 Октябрь 2024
на канале: Melvin L
4,095
27

Tip: Infer table schema automatically using Impala (using CREATE ..LIKE PARQUET)

Comparing Hive vs Impala options

Option 1: Using Hive- Manually build the table schema with all the column details
---------------------
CREATE EXTERNAL TABLE person (name String, age Int, sex String)
STORED as PARQUET
LOCATION '/tmp/person'

Option 2: Using Impala - Automatically infer table schema
------------------------
CREATE EXTERNAL TABLE person2 LIKE PARQUET '/tmp/person/part-r-00000-8a445cfc-eab6-41e6-8c33-40fe8aa6600d.gz.parquet'
STORED AS PARQUET
LOCATION '/tmp/person';