Amazon EMR 6.12.0 - Hive release notes - Amazon EMR

Amazon EMR 6.12.0 - Hive release notes

Amazon EMR 6.12.0 - Hive changes

Type Description
Improvement Added Support For JDK 11 and JDK 17 Runtime
Improvement Add support to query case sensitive and reserved keyword column names when using S3 Select. To use it, define table property in format "s3select.column.mapping" = "column1:fieldName1, column2:fieldName2,..."
Improvement HIVE-23133: Numeric operations can have different result across hardware archs
Improvement HIVE-27145: Use StrictMath for remaining Math functions as followup of HIVE-23133
Bug Fix Fix wildcard incompatibility in get_partitions_by_filter and get_num_partitions_by_filter HMS APIs caused by porting HIVE-22900 in EMR Hive 6.4.0
Bug Fix HIVE-26736: Authorization failure for nested Views having WITH clause
Bug Fix HIVE-22416: MR-related operation logs missing when parallel execution is enabled
Bug Fix HIVE-19653: Incorrect predicate pushdown for groupby with grouping sets
Bug Fix HIVE-22094: queries failing with ClassCastException: hive.ql.exec.vector.DecimalColumnVector cannot be cast to hive.ql.exec.vector.Decimal64ColumnVector
Bug Fix HIVE-26340: Vectorized PTF operator fails if query has upper case window function
Bug Fix HIVE-26184: COLLECT_SET with GROUP BY is very slow when some keys are highly skewed
Bug Fix HIVE-26373: ClassCastException when reading timestamps from HBase table with Avro data
Bug Fix HIVE-26388: ClassCastException when there is non string type column in source table of CTAS query Upgrade HIVE-26172: Hive - Upgrade Ant to 1.10.11 due to CVE-2021-36373 and CVE-2021-36374
Bug Fix HIVE-26114: Fix jdbc connection hiveserver2 using dfs command with prefix space will cause exceptio
Bug Fix HIVE-26396: The trunc function has a problem with precision interception and the result has many 0
Bug Fix HIVE-26446: HiveProtoLoggingHook fails to populate TablesWritten field for partitioned tables.
Bug Fix HIVE-26639: ConstantVectorExpression and ExplainTask shouldn't rely on default charset
Bug Fix HIVE-22670: ArrayIndexOutOfBoundsException when vectorized reader is used for reading a parquet file
Bug Fix HIVE-23607: Permission Issue: Create view on another view succeeds but alter view fails
Bug Fix HIVE-25498: Query with more than 31 count distinct functions returns wrong result
Bug Fix HIVE-25780: DistinctExpansion creates more than 64 grouping sets II
Bug Fix HIVE-23868: Windowing function spec: support 0 preceeding/following
Bug Fix HIVE-24539: OrcInputFormat schema generation should respect column delimiter
Bug Fix HIVE-23476: LLAP: Preallocate arenas for mmap case as well
Bug Fix HIVE-25806: Possible leak in LlapCacheAwareFs - Parquet, LLAP IO
Bug Fix HIVE-23498: Disable HTTP Trace method on ThriftHttpCliService
Bug Fix HIVE-25729: ThriftUnionObjectInspector should be notified when fully inited
Bug Fix HIVE-23846: Avoid unnecessary serialization and deserialization of bitvectors
Bug Fix HIVE-24233: except subquery throws nullpointer with cbo disabled
Bug Fix HIVE-24276: HiveServer2 loggerconf jsp Cross-Site Scripting (XSS) Vulnerability
Bug Fix HIVE-25721: Outer join result is wrong
Bug Fix HIVE-25223: Select with limit returns no rows on non native table
Bug Fix HIVE-25794: CombineHiveRecordReader: log statements in a loop leads to memory pressure
Bug Fix HIVE-23602: Use Java Concurrent Package for Operation Handle Set
Bug Fix HIVE-24045: No logging related to when default database is created
Bug Fix HIVE-24305: avro decimal schema is not properly populating scale/precision if value is enclosed in quote
Bug Fix HIVE-25844: Exception deserialization error-s may cause beeline to terminate immediately
Bug Fix HIVE-25040: Drop database cascade cannot remove persistent functions
Bug Fix HIVE-23501: AOOB in VectorDeserializeRow when complex types are converted to primitive types
Bug Fix HIVE-23704: Thrift HTTP Server Does Not Handle Auth Handle Correctly
Bug Fix HIVE-23529: CTAS is broken for uniontype when row_deserialize
Bug Fix HIVE-24144: getIdentifierQuoteString in HiveDatabaseMetaData returns incorrect value
Bug Fix HIVE-23850: Allow PPD when subject is not a column with grouping sets present
Bug Fix HIVE-24036: Kryo Exception while serializing plan for getSplits UDF call
Bug Fix HIVE-25919: ClassCastException when pushing boolean column predicate in HBaseStorageHandler
Bug Fix HIVE-25261: RetryingHMSHandler should wrap the MetaException with short description of the target
Bug Fix HIVE-24792: Potential thread leak in Operation
Bug Fix HIVE-23409: If TezSession application reopen fails for Timeline service down, default TezSession from SessionPool is closed after a retry
Bug Fix HIVE-23615: Do not deference null pointers in Beeline Commands Class
Bug Fix HIVE-24849: Create external table socket timeout when location has large number of files (affects 3.1.2)
Bug Fix HIVE-24193: Select query on renamed hive acid table does not produce any output
Bug Fix HIVE-25209: SELECT query with SUM function producing unexpected result
Bug Fix HIVE-23666: checkHashModeEfficiency is skipped when a groupby operator doesn't have a grouping set
Bug Fix HIVE-23873: Querying Hive JDBCStorageHandler table fails with NPE when CBO is off
Bug Fix HIVE-24149: HiveStreamingConnection doesn't close HMS connection
Bug Fix HIVE-25561: Killed task should not commit file. (affects 2.x and 3.x versions)
Bug Fix HIVE-25683: Close reader in AcidUtils.isRawFormatFile
Bug Fix HIVE-24294: TezSessionPool sessions can throw AssertionError
Bug Fix HIVE-24182: Ranger authorization issue with permanent UDFs
Bug Fix HIVE-22805: Vectorization with conditional array or map is not implemented and throws an error
Bug Fix HIVE-22828: Decimal64: NVL and CASE statements implicitly convert decimal64 to 128
Bug Fix HIVE-21398: Columns which has estimated statistics should not be considered as unique keys
Bug Fix HIVE-22490: Adding jars with special characters in their path throws error
Bug Fix HIVE-22700: Compactions may leak memory when unauthorized
Bug Fix HIVE-22053: Function name is not normalized when creating function
Bug Fix HIVE-22595: Dynamic partition inserts fail on Avro table table with external schema
Bug Fix HIVE-21795: Rollup summary row might be missing when a mapjoin is happening on a partitioned table
Bug Fix HIVE-22987: ClassCastException in VectorCoalesce when DataTypePhysicalVariation is null
Bug Fix HIVE-22814: ArrayIndexOutOfBound in the vectorization getDataTypePhysicalVariation
Bug Fix HIVE-22523: The error handler in LlapRecordReader might block if its queue is full
Bug Fix HIVE-21796: ArrayWritableObjectInspector.equals can take O(2^nesting_depth) time
Bug Fix HIVE-22929: Performance: quoted identifier parsing uses throwaway Regex via String.replaceAll()
Bug Fix HIVE-21641: Llap external client returns decimal columns in different precision/scale as compared to beeline
Bug Fix HIVE-22207: Tez: SplitGenerator throws NumberFormatException when "dfs.block.size" on cluster is "128m"
Bug Fix HIVE-22114: Insert query for partitioned insert only table failing when all buckets are empty
Bug Fix HIVE-22165: Synchronisation introduced by HIVE-14296 on SessionManager.closeSession causes high latency in a busy hive server
Bug Fix HIVE-22744: TezTask for the vertex with more than one outedge should have proportional sort memory
Bug Fix HIVE-22072: Altering table to make a column change does not update constraints references
Bug Fix HIVE-22075: Fix the max-reducers=1 regression from HIVE-14200
Bug Fix HIVE-22527: Hive on Tez : Job of merging small files will be submitted into another queue (default queue)
Bug Fix HIVE-22816: QueryCache: Queries using views can have them cached after CTE expansion
Bug Fix HIVE-22733: After disable operation log property in hive, still HS2 saving the operation log
Bug Fix HIVE-22699: Mask UDFs should mask numeric value 0
Bug Fix HIVE-23356: Hash aggregation is always disabled while processing querys with grouping sets expressions.
Bug Fix HIVE-21568: HiveRelOptUtil.isRowFilteringPlan should skip Project
Bug Fix HIVE-21760: Sharedwork optimization should be bypassed for SMB joins
Bug Fix HIVE-22712: ReExec Driver execute submit the query in default queue irrespective of user defined queue
Bug Fix HIVE-21397: BloomFilter for hive Managed [ACID] table does not work as expected
Bug Fix HIVE-23011: Shared work optimizer should check residual predicates when comparing joins
Bug Fix HIVE-21412: PostExecOrcFileDump doesn't work with ACID tables
Bug Fix HIVE-22201: ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected
Bug Fix HIVE-21971: HS2 leaks classloader due to `ReflectionUtils::CONSTRUCTOR_CACHE` with temporary functions + GenericUDF
Bug Fix HIVE-21368: Vectorization: Unnecessary Decimal64 ->HiveDecimal conversion
Bug Fix HIVE-25416: Hive metastore memory leak because datanucleus-api-jdo bug
Bug Fix HIVE-22219: Bringing a node manager down blocks restart of LLAP service
Bug Fix HIVE-21793: CBO retrieves column stats even if hive.stats.fetch.column.stats is set to false
Bug Fix HIVE-22163: CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
Bug Fix HIVE-18735: Create table like loses transactional attribute
Bug Fix HIVE-22433: Hive JDBC Storage Handler: Incorrect results fetched from BOOLEAN and TIMESTAMP DataType From JDBC Data Source
Bug Fix HIVE-19430: ObjectStore.cleanNotificationEvents OutOfMemory on large number of pending events
Bug Fix HIVE-20785: Wrong key name in the JDBC DatabaseMetaData.getPrimaryKeys method
Bug Fix HIVE-16116: Beeline throws NPE when beeline.hiveconfvariables={} in
Bug Fix HIVE-20066: is compared to full principal
Bug Fix HIVE-20489: Explain plan of query hangs
Bug Fix HIVE-21033: Forgetting to close operation cuts off any more HiveServer2 output
Bug Fix HIVE-19888: Misleading "METASTORE_FILTER_HOOK will be ignored" warning from SessionState
Bug Fix HIVE-20303: INSERT OVERWRITE TABLE db.table PARTITION (...) IF NOT EXISTS throws InvalidTableException
Bug Fix HIVE-16144: CompactionInfo doesn't have equals/hashCode but used in Set
Bug Fix HIVE-20818: Views created with a WHERE subquery will regard views referenced in the subquery as direct input
Bug Fix HIVE-21005: LLAP: Reading more stripes per-split leaks ZlibCodecs
Bug Fix HIVE-20771: LazyBinarySerDe fails on empty structs.
Bug Fix HIVE-18852: Misleading error message in alter table validation
Bug Fix HIVE-21124: HPL/SQL does not support the CREATE TABLE LIKE statement
Bug Fix HIVE-20935: Upload of llap package tarball fails in EC2 causing LLAP service start failure
Bug Fix HIVE-20409: Hive ACID: Update/delete/merge does not clean hdfs staging directory
Bug Fix HIVE-20570: Union ALL with hive.optimize.union.remove=true has incorrect plan
Bug Fix HIVE-20421: Illegal character entity '\b' in hive-default.xml.template
Bug Fix HIVE-19133: HS2 WebUI phase-wise performance metrics not showing correctly
Bug Fix HIVE-18977: Listing partitions returns different results with JDO and direct SQL
Bug Fix HIVE-20034: Roll back MetaStore exception handling changes for backward compatibility
Bug Fix HIVE-20672: Logging thread in LlapTaskSchedulerService should report every fixed interval
Bug Fix HIVE-12812: Enable mapred.input.dir.recursive by default to support union with aggregate function
Bug Fix HIVE-20147: Hive streaming ingest is contented on synchronized logging
Bug Fix HIVE-19203: Thread-Safety Issue in HiveMetaStore
Bug Fix HIVE-20091: Tez: Add security credentials for FileSinkOperator output
Bug Fix HIVE-16906: Hive ATSHook should check for yarn.timeline-service.enabled before connecting to ATS
Bug Fix HIVE-20714: SHOW tblproperties for a single property returns the value in the name column
Bug Fix HIVE-24730: Shims classes override values from hive-site.xml and tez-site.xml silently
Bug Fix HIVE-22055: select count gives incorrect result after loading data from text file