How to use DuckDB with large datasets
Admin
Below is a single unified DuckDB instruction set combining both versions into one clean, consistent workflow for large radio astronomy event datasets. Assumes columns: Unified DuckDB Workflow for Large Radio…
Uncategorized
Why am I unable to run JAVA muon data processing program on my Windows 10 computer, with the following error message appearing: UnsupportedClassVersionError
The key part of the error is: UnsupportedClassVersionError ...has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime…
Development of John’s scripts to analyse PicoMuon stand alone data during transatlantic flight
Hi Andy, Good to see you got the program running, at least on one file. Interesting that, for the top and bottom detectors, the measured standard deviations are lower than…
How to reduce the quantity of data recorded on portable Picomuon detector during a Cubesat in a Box “launch” such a s a transatlantic flight from Europe to USA
For a portable picoMuon detector it may be helpful to reduce the amount of data recorded. By changing events_all=True to False only coincident events will be written to the sd…
Using digital TV channels for radio meteor scatter radio astronomy observations in USA
In our discussion today regarding the use in N.A. of DTV station signals for meteor scatter illumination, in lieu of the lack of similar signals as the French GRAVES or…
JAVA muon data processing program on Windows 10 causes “UnsupportedClassVersionError”.
The key part of the error is: UnsupportedClassVersionError ...has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime…
What free spreadsheet software is available to process very large PicoMuon data CSV files, which are too large for Microsoft Excel?
If you need a free Windows spreadsheet that can cope with very large spreadsheets, these are the best choices: 1. LibreOffice Calc (recommended by ChatGPT but I had difficulties installing…
How to open DuckDB (command line database editor) which works with very large PicoMuon data files that are too big for Excel
If you’re asking how to open DuckDB on Windows, it depends on what you downloaded. If you have duckdb.exe, simply double-click it. A command window will open with a D>…
How to cut a file in half with DuckDB, and also how to split by date?
One of the nicest features of DuckDB is that you don’t need to import a CSV before working with it. For example, if your file is called large.csv: View the…
How to combine separate date and time columns into a single date-time column them filter by ID column = C then create Pivot table of events from Event column (where ID column = C) with horizontal access as hourly.
You can do all of that cleanly in DuckDB using one SQL pipeline: combine date+time → filter by ID → build hourly pivot. I’ll assume your CSV has columns like:…