Issue: Intermittent "missing value where TRUE/FALSE needed" error in g.getmeta during CSV processing

[umbertomazzucchelli]

Hi all,

I am currently trying to process accelerometer data collected on participants for a month straight. It was collected with Fossil Gen 4/5 smartwatches and then converted with Python to CSV format.

System Information

• OS: Ubuntu 24.04
• RAM: 96 GB
• GGIR Version: 3.2.10

File Characteristics Comparison

Key differences in data gaps:

• Working file: Max gap = 66,044s (18.35h), 62 gaps > 1min
• Failing file: Max gap = 16,323s (4.53h), 46 gaps > 1min

Current GGIR Script

GGIR(
    # Core settings
    datadir = temp_data_dir,
    studyname = paste0(participant_id, "_month_", month_number),
    outputdir = participant_output_dir,
    # Processing modes
    mode = c(1:6),
    do.report = c(2, 4, 5, 6),
    # Raw data configuration for CSV files
    rmc.firstrow.acc = 2,
    rmc.col.acc = 2:4,           # x, y, z columns
    rmc.col.time = 1,            # timestamp column
    rmc.sf = 50,                 # sampling frequency
    rmc.unit.acc = "g",          # acceleration unit
    rmc.format.time = "%Y-%m-%d %H:%M:%OS", 
    # Metrics to calculate
    do.enmo = TRUE,
    do.anglez = TRUE,
    do.cal = TRUE,
    do.imp = TRUE,
    # Processing settings
    chunksize = 1,
    minimumFileSizeMB = 0,
    print.filename = TRUE,
    printsummary = TRUE,
    do.parallel = TRUE,
    overwrite = TRUE,
    # Time windows and criteria
    qwindow = c(0, 24),
    hrs.del.start = 0,
    hrs.del.end = 0,
    includedaycrit = 16,
    winhr = c(5, 10),
    M5L5res = 10,
    # Physical activity thresholds
    mvpathreshold = 100,
    mvpadur = c(1, 5, 10),
    boutcriter = 0.8,
    threshold.lig = 40,
    threshold.mod = 100,
    threshold.vig = 400,
    # Sleep detection
    timethreshold = 5,
    anglethreshold = 5,
    HASPT.algo = "HDCZA",
    acc.metric = "ENMO",
    ignorenonwear = TRUE,
    # Part 4 (sleep analysis)
    excludefirstlast = FALSE,
    includenightcrit = 16,
    def.noc.sleep = 1,
    do.visual = TRUE,
    outliers.only = FALSE,
    criterror = 3,
    relyonguider = FALSE,
    sleepwindowType = "SPT",
    # Sleep log configuration (only if file exists)
    loglocation = if (!is.null(sleep_log_file) && !is.na(sleep_log_file)) sleep_log_file else "",
    # Part 5 (physical activity bouts)
    boutcriter.in = 0.9,
    boutcriter.lig = 0.8,
    boutcriter.mvpa = 0.8,
    boutdur.in = c(10, 20, 30),
    boutdur.lig = c(1, 5, 10),
    boutdur.mvpa = c(1, 5, 10),
    timewindow = c("MM", "WW"),
    save_ms5rawlevels = TRUE,
    save_ms5raw_format = "csv",
    frag.metrics = "all",
    part5_agg2_60seconds = FALSE,
    # Part 6 (circadian rhythm)
    part6CR = TRUE,
    part6Window = c("start", "end"),
    # Report generation
    visualreport = TRUE,
    dofirstpage = TRUE,
    viewingwindow = 1,
    # General parameters
    windowsizes = c(5, 900, 3600),
    maxdur = 365,
    strategy = 1,
    minimum_MM_length.part5 = 1,
    ndayswindow = 7
  )

Problem Description

The script runs intermittently - sometimes it processes without any issues, while other times it crashes during g.getmeta with the following error:

Error in GGIR processing: missing value where TRUE/FALSE needed

Troubleshooting Attempts

• Verified CSV format consistency across files
• Checked for missing values/empty rows (both files have 0 missing values)
• Confirmed timestamp format matches specification (%Y-%m-%d %H:%M:%OS)
• Analyzed data integrity - both files have duplicate timestamps but no missing data

Main Questions

1. Is there a file size/row count threshold for GGIR CSV processing?

2. Could the issue be related to:

   • Memory limitations during g.getmeta processing of larger datasets?
   • Integer overflow or indexing issues with very large row counts?
   • Chunk processing limits when chunksize = 1?

3. Is there a way to get more detailed error logging from g.getmeta to pinpoint exactly where the TRUE/FALSE evaluation fails and why?

Any insights or suggestions would be greatly appreciated!

[vincentvanhees]

Hi Umberto,

1. The number of rows read are controlled by parameter chunksize, where value 1 is default and reflects approximately 24 hours of raw data. Chunksize itself is contrainted to any value above 0.5. To minimise the role of memory shortage try processing serially with do.parallel = FALSE,. If that solves the error then indicates a memory issue.
2. I doubt it, because GGIR has been used on various datasets with high resolution data in .csv format, unless your data has a much large number of columns with data GGIR does not need but loads and/or a much higher sampling rate than what GGIR is mostly used with (100 or less Hertz). Is either of these conditions the case?
3. With do.parallel = FALSE, you should get better error messages.

To better help you it would be good to know:

• In which GGIR part (1, 2, 3, 4, 5, or 6) the code generates an error?
• Which specific function triggers the error?

[umbertomazzucchelli]

Hi Vincent,

I tried to run it again with do.parallel = FALSE, but I am getting the same error. It's happening during the g.getmeta function call, during part 1:

Extract signal features (metrics) with the g.getmeta function:

Loading chunk: 1 2 3 4 5 6 7 8 9 10 11 12Error in GGIR processing: missing value where TRUE/FALSE needed

I doubt it, because GGIR has been used on various datasets with high resolution data in .csv format, unless your data has a much large number of columns with data GGIR does not need but loads and/or a much higher sampling rate than what GGIR is mostly used with (100 or less Hertz). Is either of these conditions the case?

No, the data was collected at 50Hz. It's just a very long CSV, but similar files from the study run smoothly, so you're right.

Thank you for your help.

[vincentvanhees]

Hi Umberto,
I am happy to help investigate further if you or your employer is willing to hire me for a short consultancy project to look into this. Please contact me via email if you are interested (v.vanhees at accelting dot com)

GGIR is free software. If the community wants me to keep responding to questions and doing essential maintenance then I have no choice other than to ask for payment for my time where possible.

[alice-chv]

Hi,
I'm having the same problem.
I converted some files from SensMotion sensors to .csv.
For some files, it works perfectly fine, but for others, I'm getting the same error in part1 g.getmeta:

Error in if (first_deltatime >= k) { :
Missing value where TRUE / FALSE needed

Do you have any suggestions for resolving this issue you've encountered?
Thank you!
Alice

[umbertomazzucchelli]

Hi @alice-chv,

I ended up solving the issue by skipping the timegaps imputation in GGIR and doing it manually on the data, before feeding it into GGIR.
My script does the following:

1. It calculates the time difference between consecutive timestamps. It flags any difference greater than or equal to 0.25 seconds (this depends on your sampling frequency).
2. For each gap, it calculates the number of missing samples required to fill the gap at a 50 Hz sampling frequency (n_imputed_samples <- round(gap_duration * sf)).
3. It fills the missing samples by repeating the precise accelerometer XYZ values of the last known valid reading immediately preceding the gap.
4. It calculates the vector magnitude of the carried-forward row. If the magnitude deviates from $1g$ by more than $0.005g$, it divides the XYZ axes by the magnitude to force the vector exactly to $1g$.
5. It creates a new sequence of POSIX timestamps for the imputed rows, incrementing exactly by $\frac{1}{50}$ of a second ($0.02s$).
6. It stores alternating chunks of valid original data and newly generated imputed data into a list, which is combined back into a single, continuous data frame (do.call(rbind, new_rows_list)) before saving it to a temporary CSV for GGIR to process.

It's not a perfect solution, but it resolved the error. I was able to process my entire dataset in this way. Feel free to contact me and I will be happy to share my script with you!

Umberto

[vincentvanhees]

@umbertomazzucchelli GGIR is open-source software and by that depends on the user community to help improve the code.
Please share your fix publicly instead of this narrative summary and expecting people to contact you.

Further, it would be even more appreciated by the community if you could make an effort to help improve GGIR such that the script is not even needed. You are also benefiting from the many researchers who have taken the effort to report and help improve past problems with GGIR. So, you may want to see it as a way of returning the favour.

[umbertomazzucchelli]

@vincentvanhees , happy to share the script publicly — I initially offered it privately only because it is an ad-hoc workaround rather than a polished contribution, and I did not want to clutter the thread.

To clarify why I haven't submitted a pull request: my work is primarily Python-based, and I was running GGIR to support a collaborator rather than as a core part of my own work. Beyond that, I was never able to fully identify the root cause of the bug within the GGIR source, as my R proficiency is not sufficient to trace it through the codebase. The best I could do was build a preprocessing workaround with the help of Claude that sidesteps the issue entirely.

I take your point about giving back to the community. I opened a detailed bug report and I'm now glad to share the full script below so others can use it directly. Hopefully, the root cause can be addressed in a future GGIR release. If it would be helpful, I'm also happy to open a formal issue to make the bug more trackable.

Thank you for all the work you do on GGIR.

# GGIR Monthly Processing Script
# Usage: Rscript ggir_monthly.R <month_folder_path> <output_dir> [timezone]

# Get command line arguments
args <- commandArgs(trailingOnly = TRUE)

if (length(args) < 2 || length(args) > 3) {
  cat("Usage: Rscript ggir_monthly.R <month_folder_path> <output_dir> [timezone]\n")
  cat("Example: Rscript ggir_monthly.R /path/to/participant/month_12_2022-07-18_2022-08-03 /path/to/output EDT\n")
  cat("         Rscript ggir_monthly.R /path/to/participant/month_12_2022-07-18_2022-08-03 /path/to/output\n")
  cat("\nSupported timezones: PST, PDT, MST, MDT, CST, CDT, EST, EDT, HST, etc.\n")
  quit(status = 1)
}

month_folder_path <- args[1]
output_dir <- args[2]
timezone_abbr <- if (length(args) >= 3) args[3] else NULL

# Debug utilities (enable with GGIR_DEBUG=true)
debug_enabled <- tolower(Sys.getenv("GGIR_DEBUG", unset = "false")) %in% c("1", "true", "t", "yes", "y")
dbg <- function(...) { if (debug_enabled) cat("[DEBUG]", ..., "\n") }

# Convert timezone abbreviation to R timezone name
# Map all timezone abbreviations found in the data (PDT, EDT, PST, EST, CDT, CST, MDT, MST, HST)
convert_tz_abbr_to_r <- function(tz_abbr) {
  if (is.null(tz_abbr) || is.na(tz_abbr) || tz_abbr == "") {
    return("UTC")
  }

  tz_map <- list(
    "PST" = "America/Los_Angeles",  # Pacific Standard Time
    "PDT" = "America/Los_Angeles",  # Pacific Daylight Time
    "MST" = "America/Denver",       # Mountain Standard Time
    "MDT" = "America/Denver",       # Mountain Daylight Time
    "CST" = "America/Chicago",      # Central Standard Time
    "CDT" = "America/Chicago",      # Central Daylight Time
    "EST" = "America/New_York",     # Eastern Standard Time
    "EDT" = "America/New_York",     # Eastern Daylight Time
    "HST" = "Pacific/Honolulu"      # Hawaii Standard Time
  )

  tz_name <- tz_map[[toupper(tz_abbr)]]
  if (is.null(tz_name)) {
    cat("Warning: Unknown timezone abbreviation '", tz_abbr, "', using UTC\n", sep = "")
    return("UTC")
  }

  return(tz_name)
}

# Set the timezone for this processing run
participant_tz <- convert_tz_abbr_to_r(timezone_abbr)
cat("Using timezone:", participant_tz, if (!is.null(timezone_abbr)) paste0(" (", timezone_abbr, ")") else "", "\n")

# Extract information from folder path
folder_name <- basename(month_folder_path)

# Parse month information from folder name
month_parts <- strsplit(folder_name, "_")[[1]]
if (length(month_parts) < 4) {
  cat("Error: Month folder name must follow pattern month_<n>_<start>_<end>\n")
  quit(status = 1)
}
month_number <- month_parts[2]
start_date <- month_parts[3]
end_date <- month_parts[4]

# Discover participant ID based on data file naming
data_dir <- file.path(month_folder_path, "data")
if (!dir.exists(data_dir)) {
  cat("Error: Data directory not found:", data_dir, "\n")
  quit(status = 1)
}

csv_pattern <- paste0("_month_", month_number, "\\.csv$")
csv_candidates <- list.files(data_dir, pattern = csv_pattern, full.names = FALSE)
if (length(csv_candidates) == 0) {
  cat("Error: No CSV matching pattern '*_month_", month_number, ".csv' found in", data_dir, "\n", sep = "")
  quit(status = 1)
}

data_basename <- csv_candidates[1]
participant_id <- sub(csv_pattern, "", data_basename)
dbg("Detected participant ID:", participant_id)

# Create output directory for this participant and month FIRST
participant_output_dir <- file.path(output_dir, participant_id, folder_name)
dir.create(participant_output_dir, recursive = TRUE, showWarnings = FALSE)

# Load GGIR
library(GGIR)

# Construct file paths
data_file <- file.path(data_dir, data_basename)
sleep_log_file <- list.files(month_folder_path, pattern = ".*sleep_log.*\\.csv$", full.names = TRUE)[1]

# Check if files exist
if (!file.exists(data_file)) {
  cat("Error: Data file not found:", data_file, "\n")
  quit(status = 1)
}

if (is.na(sleep_log_file) || !file.exists(sleep_log_file)) {
  cat("Warning: Sleep log file not found in", month_folder_path, "\n")
  sleep_log_file <- NULL
}

# Create a temporary directory with just the data file for GGIR
temp_data_dir <- file.path(participant_output_dir, "temp_data")
dir.create(temp_data_dir, recursive = TRUE, showWarnings = FALSE)

# CRITICAL FIX: Pre-process data to handle problematic values that cause logical errors
cat("Pre-processing data to fix potential logical operation issues...\n")

# Read and clean the data
raw_data <- read.csv(data_file)
cat("Original data rows:", nrow(raw_data), "\n")

# Fix potential issues that cause logical errors in GGIR:
# 1. Ensure no infinite values
infinite_count <- sum(is.infinite(as.matrix(raw_data[, 2:4])))
if (infinite_count > 0) {
  cat("Fixed", infinite_count, "infinite values\n")
  raw_data[, 2:4][is.infinite(as.matrix(raw_data[, 2:4]))] <- 0
}

# 2. Ensure no NA values in accelerometer data
na_count <- sum(is.na(raw_data[, 2:4]))
if (na_count > 0) {
  cat("Fixed", na_count, "NA values in accelerometer columns\n")
  raw_data[, 2:4][is.na(raw_data[, 2:4])] <- 0
}

# 2b. Check for NA values in timestamp column (critical for GGIR)
timestamp_na_count <- sum(is.na(raw_data[, 1]))
if (timestamp_na_count > 0) {
  cat("ERROR: Found", timestamp_na_count, "NA values in timestamp column\n")
  # Remove rows with NA timestamps as they cannot be processed
  raw_data <- raw_data[!is.na(raw_data[, 1]), ]
  cat("Removed rows with NA timestamps. Remaining rows:", nrow(raw_data), "\n")
}

# 3. Fix extreme values that might cause mathematical issues
extreme_threshold <- 20  # g values beyond human possibility
extreme_fixed <- 0
for (col in 2:4) {
  extreme_indices <- which(abs(raw_data[, col]) > extreme_threshold)
  if (length(extreme_indices) > 0) {
    extreme_fixed <- extreme_fixed + length(extreme_indices)
    raw_data[extreme_indices, col] <- sign(raw_data[extreme_indices, col]) * extreme_threshold
  }
}
if (extreme_fixed > 0) {
  cat("Fixed", extreme_fixed, "extreme values (>", extreme_threshold, "g)\n")
}

# 4. Ensure magnitude is never zero (can cause division by zero issues in GGIR)
magnitude <- sqrt(raw_data[, 2]^2 + raw_data[, 3]^2 + raw_data[, 4]^2)
zero_magnitude_indices <- which(magnitude < 1e-6)
if (length(zero_magnitude_indices) > 0) {
  cat("Fixed", length(zero_magnitude_indices), "near-zero magnitude values\n")
  # Set to minimal valid magnitude to prevent division by zero
  raw_data[zero_magnitude_indices, 2] <- 0.001
  raw_data[zero_magnitude_indices, 3] <- 0.001
  raw_data[zero_magnitude_indices, 4] <- 0.001
}

# Final validation: Ensure no NA values remain before GGIR processing
final_na_check <- sum(is.na(raw_data))
if (final_na_check > 0) {
  cat("WARNING: Still found", final_na_check, "NA values after cleaning\n")
  cat("Performing final cleanup...\n")
  # Remove any rows with any NA values
  complete_rows <- complete.cases(raw_data)
  removed_count <- sum(!complete_rows)
  if (removed_count > 0) {
    cat("Removing", removed_count, "incomplete rows\n")
    raw_data <- raw_data[complete_rows, ]
  }
}

# Validate and standardize timestamp column
timestamp_col_name <- names(raw_data)[1]
raw_data[[timestamp_col_name]] <- trimws(raw_data[[timestamp_col_name]])
parsed_time <- as.POSIXct(raw_data[[timestamp_col_name]], format = "%Y-%m-%d %H:%M:%OS", tz = participant_tz)

if (debug_enabled) {
  head_ts <- head(raw_data[[timestamp_col_name]], 10)
  tail_ts <- tail(raw_data[[timestamp_col_name]], 10)
  head_ok <- as.integer(!is.na(as.POSIXct(head_ts, format = "%Y-%m-%d %H:%M:%OS", tz = participant_tz)))
  tail_ok <- as.integer(!is.na(as.POSIXct(tail_ts, format = "%Y-%m-%d %H:%M:%OS", tz = participant_tz)))
  dbg("Timestamp parse check head:", paste(head_ok, collapse = ","))
  dbg("Timestamp parse check tail:", paste(tail_ok, collapse = ","))
}

invalid_timestamp_indices <- which(is.na(parsed_time))
if (length(invalid_timestamp_indices) > 0) {
  cat("Removed", length(invalid_timestamp_indices), "rows with invalid timestamps\n")
  raw_data <- raw_data[-invalid_timestamp_indices, , drop = FALSE]
  parsed_time <- parsed_time[-invalid_timestamp_indices]
}

if (nrow(raw_data) == 0) {
  cat("WARNING: No valid timestamp data remaining; skipping month.\n")
  quit(status = 0)
}

order_idx <- order(parsed_time)
if (!all(order_idx == seq_along(parsed_time))) {
  raw_data <- raw_data[order_idx, , drop = FALSE]
  parsed_time <- parsed_time[order_idx]
  cat("Sorted data by timestamp\n")
}

duplicate_timestamps <- duplicated(parsed_time)
if (any(duplicate_timestamps)) {
  cat("Removed", sum(duplicate_timestamps), "duplicate timestamps\n")
  raw_data <- raw_data[!duplicate_timestamps, , drop = FALSE]
  parsed_time <- parsed_time[!duplicate_timestamps]
}

recording_duration_hours <- as.numeric(difftime(max(parsed_time), min(parsed_time), units = "hours"))
if (is.na(recording_duration_hours) || recording_duration_hours < 16) {
  cat("WARNING: Recording duration too short (", round(recording_duration_hours, 2), " hours); skipping month.\n", sep = "")
  quit(status = 0)
}

# Pre-impute timegaps to avoid GGIR chunk boundary bug
# We'll disable GGIR's imputeTimegaps later to bypass the buggy code
cat("Pre-imputing time gaps (to avoid GGIR bug)...\n")
sf <- 50  # sampling frequency
min_gap_seconds <- 0.25  # same as GGIR's k parameter
deltatime <- as.numeric(diff(parsed_time))
gap_indices <- which(deltatime >= min_gap_seconds)

if (length(gap_indices) > 0) {
  cat("Found", length(gap_indices), "time gaps >=", min_gap_seconds, "s\n")

  # Build new data frame with imputed values
  new_rows_list <- list()
  current_idx <- 1
  total_imputed <- 0

  for (gap_idx in gap_indices) {
    # Add all rows before the gap
    if (current_idx <= gap_idx) {
      new_rows_list[[length(new_rows_list) + 1]] <- raw_data[current_idx:gap_idx, ]
    }

    # Calculate how many samples to impute
    gap_duration <- deltatime[gap_idx]
    n_imputed_samples <- round(gap_duration * sf)

    if (n_imputed_samples > 1) {
      # Create imputed rows by repeating the last known value
      last_known_row <- raw_data[gap_idx, ]
      imputed_rows <- last_known_row[rep(1, n_imputed_samples - 1), ]

      # Normalize acceleration to 1g if needed (GGIR does this)
      accel_cols <- 2:4
      magnitude <- sqrt(sum(last_known_row[, accel_cols]^2))
      if (abs(magnitude - 1) > 0.005) {  # only if deviation > 5mg
        imputed_rows[, accel_cols] <- imputed_rows[, accel_cols] / magnitude
      }

      # Generate timestamps for imputed samples
      start_time <- parsed_time[gap_idx]
      imputed_times <- start_time + seq(1/sf, by = 1/sf, length.out = n_imputed_samples - 1)
      imputed_rows[[timestamp_col_name]] <- format(imputed_times, "%Y-%m-%d %H:%M:%OS3")

      new_rows_list[[length(new_rows_list) + 1]] <- imputed_rows
      total_imputed <- total_imputed + (n_imputed_samples - 1)
    }

    current_idx <- gap_idx + 1
  }

  # Add remaining rows
  if (current_idx <= nrow(raw_data)) {
    new_rows_list[[length(new_rows_list) + 1]] <- raw_data[current_idx:nrow(raw_data), ]
  }

  # Combine all rows
  raw_data <- do.call(rbind, new_rows_list)
  rownames(raw_data) <- NULL

  # Reparse timestamps
  parsed_time <- as.POSIXct(raw_data[[timestamp_col_name]], format = "%Y-%m-%d %H:%M:%OS", tz = participant_tz)

  cat("Imputed", total_imputed, "samples. Total rows now:", nrow(raw_data), "\n")
} else {
  cat("No time gaps found, no imputation needed\n")
}

raw_data[[timestamp_col_name]] <- format(parsed_time, "%Y-%m-%d %H:%M:%OS3")


# Final row count validation
if (nrow(raw_data) < 100) {
  cat("WARNING: Too few valid rows remaining:", nrow(raw_data), "- skipping month without running GGIR.\n")
  quit(status = 0)
}

cat("Final data validation: ", nrow(raw_data), "rows, all clean\n")

# Create cleaned data file for GGIR processing
cleaned_data_file <- file.path(temp_data_dir, paste0("cleaned_", basename(data_file)))
write.csv(raw_data, cleaned_data_file, row.names = FALSE)
cat("Cleaned data saved for GGIR processing\n")

# Update sleeplog to reference the cleaned filename if sleeplog exists
if (!is.null(sleep_log_file) && file.exists(sleep_log_file)) {
  sleeplog_data <- read.csv(sleep_log_file)
  usable_sleep_entries <- nrow(sleeplog_data) > 0 && any(!is.na(trimws(as.character(unlist(sleeplog_data[, -1, drop = FALSE])))))

  if (usable_sleep_entries) {
    original_filename <- basename(data_file)
    cleaned_filename <- paste0("cleaned_", original_filename)
    sleeplog_data[1, 1] <- cleaned_filename

    sleeplog_dir <- file.path(participant_output_dir, "sleeplog")
    dir.create(sleeplog_dir, recursive = TRUE, showWarnings = FALSE)
    temp_sleeplog_file <- file.path(sleeplog_dir, "sleeplog.csv")
    write.csv(sleeplog_data, temp_sleeplog_file, row.names = FALSE)

    cat("Updated sleeplog ID from", original_filename, "to", cleaned_filename, "\n")
    sleep_log_file <- temp_sleeplog_file
  } else {
    cat("Sleep log contains no usable entries; ignoring sleep log for this month.\n")
    sleep_log_file <- NULL
  }
}


cat("Processing participant:", participant_id, "\n")
cat("Month:", month_number, "(", start_date, "to", end_date, ")\n")
cat("Data file:", data_file, "\n")
cat("Sleep log:", ifelse(is.null(sleep_log_file), "None", sleep_log_file), "\n")
cat("Output directory:", participant_output_dir, "\n")

# Instrument GGIR internals for debugging (prints chunk-boundary info in g.imputeTimegaps)
if (debug_enabled) {
  dbg("Enabling trace for GGIR:::g.imputeTimegaps")
  try(suppressWarnings(trace(
    what = "g.imputeTimegaps",
    where = asNamespace("GGIR"),
    tracer = quote({
      try({
        # Local variables from g.imputeTimegaps: x, sf, k, PreviousLastTime, epochsize
        t1 <- NA
        cls <- NA
        if ("time" %in% names(x)) {
          t1 <- x$time[1]
          cls <- paste(class(x$time), collapse = ";")
        }
        fdt <- NA_real_
        if (!is.null(PreviousLastTime) && ("time" %in% names(x)) && length(x$time) >= 1) {
          fdt <- diff(c(PreviousLastTime, x$time[1]))
          if (!is.numeric(fdt)) { units(fdt) <- "secs"; fdt <- as.numeric(fdt) }
        }
        cat(sprintf("[TRACE g.imputeTimegaps] nrow=%s sf=%s k=%s first_deltatime=%s class(time)=%s time[1]=%s prevLastTime=%s\n",
            NROW(x), as.character(sf), as.character(k), as.character(fdt),
            as.character(cls), as.character(t1), as.character(PreviousLastTime)))
      }, silent = TRUE)
    }),
    print = FALSE
  )), silent = TRUE)
}

# Run GGIR (without tryCatch)
GGIR(
  # Core settings
  datadir = temp_data_dir,
  studyname = paste0(participant_id, "_month_", month_number),
  outputdir = participant_output_dir,

  # Processing modes
  mode = c(1:6),
  do.report = c(2, 4, 5, 6),

  # Raw data configuration for CSV files
  rmc.firstrow.acc = 2,
  rmc.col.acc = 2:4,           # x, y, z columns
  rmc.col.time = 1,            # timestamp column
  rmc.sf = 50,                 # sampling frequency
  rmc.unit.acc = "g",          # acceleration unit
  rmc.unit.time = "POSIX",     # explicitly set POSIX format
  rmc.format.time = "%Y-%m-%d %H:%M:%OS",
  rmc.dec = ".",
  rmc.check4timegaps = FALSE,  # disable problematic timegap checking
  desiredtz = participant_tz,  # use participant's timezone for analysis
  # Metrics to calculate
  do.enmo = TRUE,
  do.anglez = TRUE,
  do.cal = TRUE,
  do.imp = TRUE,

  # Re-enable GGIR's timegap imputation to debug the NA issue
  imputeTimegaps = TRUE,  # Will trigger bug but capture debug output


  # Processing settings
  chunksize = 2,  # Reduced from 4 for smaller chunks
  minimumFileSizeMB = 0,
  print.filename = TRUE,
  printsummary = TRUE,
  do.parallel = FALSE,
  overwrite = TRUE,

  # Time windows and criteria
  qwindow = c(0, 24),
  hrs.del.start = 0,
  hrs.del.end = 0,
  includedaycrit = 16,
  winhr = c(5, 10),
  M5L5res = 10,

  # Physical activity thresholds
  mvpathreshold = 100,
  mvpadur = c(1, 5, 10),
  boutcriter = 0.8,
  threshold.lig = 40,
  threshold.mod = 100,
  threshold.vig = 400,

  # Sleep detection
  timethreshold = 5,
  anglethreshold = 5,
  HASPT.algo = "HDCZA",
  acc.metric = "ENMO",
  ignorenonwear = TRUE,

  # Part 4 (sleep analysis)
  excludefirstlast = FALSE,
  includenightcrit = 16,
  def.noc.sleep = 1,
  do.visual = TRUE,
  outliers.only = FALSE,
  criterror = 3,
  relyonguider = FALSE,
  sleepwindowType = "SPT",

  # Sleep log configuration (only if file exists)
  loglocation = if (!is.null(sleep_log_file) && !is.na(sleep_log_file)) sleep_log_file else "",

  # Part 5 (physical activity bouts)
  boutcriter.in = 0.9,
  boutcriter.lig = 0.8,
  boutcriter.mvpa = 0.8,
  boutdur.in = c(10, 20, 30),
  boutdur.lig = c(1, 5, 10),
  boutdur.mvpa = c(1, 5, 10),
  timewindow = c("MM", "WW"),
  save_ms5rawlevels = TRUE,
  save_ms5raw_format = "csv",
  frag.metrics = "all",
  part5_agg2_60seconds = FALSE,

  # Part 6 (circadian rhythm)
  part6CR = TRUE,
  part6Window = c("start", "end"),

  # Report generation
  visualreport = TRUE,
  dofirstpage = TRUE,
  viewingwindow = 1,

  # General parameters
  windowsizes = c(5, 900, 3600),
  maxdur = 365,
  strategy = 1,
  minimum_MM_length.part5 = 1,
  ndayswindow = 7
)

cat("Processing complete for", participant_id, "month", month_number, "\n")

[vincentvanhees]

Thanks, and the bug report you refer to is this discussion item?

[umbertomazzucchelli]

Yes, I was referring to this discussion.

I hadn’t opened a separate issue since I wasn’t able to fully isolate the root cause on the GGIR side and only arrived at a preprocessing workaround. If you think it would be more appropriate to track this separately as a bug, I’d be happy to open a dedicated issue with a minimal reproducible example.

[vincentvanhees]

@alice-chv if you could email me one example file and GGIR code to reproduce the issue I can have a quick look.