Numbers starting with a leading zero are treated as octal values in Bash arithmetic, causing incorrect results. To prevent this, remove leading zeros or explicitly specify the base using the 'base#' syntax.