2024-04-17T19:30:03Z INFO __main__ 304: PIPELINE VERSION Version: 1.4.1 Common instrument config version: v0.1.4 PROCESSING VERSIONS Prod: MiniReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/housekeepingL0.py Vers: 2 Prod: MaxiReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/housekeepingL0.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: Background File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: Spectra File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: Variance File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: EnergyCalibration File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: TMStatusFlareList File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: RawPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: CompressedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: SummedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: Visibility File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: Spectrogram File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: Aspect File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: MiniReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/housekeepingL1.py Vers: 2 Prod: MaxiReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/housekeepingL1.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: Background File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: Spectra File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: Variance File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: EnergyCalibration File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: TMStatusFlareList File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: RawPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: CompressedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: SummedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: Visibility File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: Spectrogram File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: Aspect File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: MiniReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/housekeepingL2.py Vers: 2 Prod: MaxiReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/housekeepingL2.py Vers: 2 Prod: Ephemeris File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/housekeepingL2.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: Background File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: Spectra File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: Variance File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: EnergyCalibration File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: TMStatusFlareList File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: LevelB File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/levelb/binary.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/lowlatency/quicklookLL.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/lowlatency/quicklookLL.py Vers: 2 2024-04-17T19:30:03Z INFO __main__ 269: CONFIG [Paths] tm_archive = /data/stix/SOLSOC/from_edds/tm/processing fits_archive = /data/stix/out/fits_v1.2.0 spice_kernels = /data/stix/spice/kernels soop_files = /data/stix/SOLSOC/from_soc [Logging] stop_on_error = False [IDLBridge] enabled = True gsw_path = /opt/STIX-GSW batchsize = 30 [Pipeline] status_server_port = 12388 parallel_batchsize_l0 = 100 parallel_batchsize_l1 = 100 parallel_batchsize_l2 = 100 log_level = INFO log_dir = /data/stix/out/fits_v1.2.0/logging error_mail_send = True error_mail_receivers = nicky.hochmuth@ateleris.ch error_mail_smpt_host = lmailer.fhnw.ch error_mail_smpt_port = 25 error_mail_sender = nicky.hochmuth@fhnw.ch start_with_unprocessed = True [Publish] db_file = /home/stixcore/soar_pub/published.sqlite include_levels = L0, L1 waiting_period = 21d target_dir = /home/stixcore/from_pub099 report_mail_receivers = nicky.hochmuth@ateleris.ch report_mail_send = True same_esa_name_dir = /data/stix/out/esa_conflicts rid_lut_file = ./stixcore/data/publish/rid_lut.csv rid_lut_file_update_url = https://datacenter.stix.i4ds.net/api/bsd/info/ batch_size = 20000 fits_dir = /data/stix/out/fits_v1.2.0 target_host = pub026.cs.technik.fhnw.ch supplement_report = /home/stixcore/soar_pub/supplement_report.csv include_products = ql,hk,sci,cal log_dir = /data/stix/out/fits_v1.2.0/logging [SOOP] endpoint = https://solarorbiter.esac.esa.int/soopkitchen/api user = smaloney password = xxxx soop_files_download = ./stixcore/data/soop 2024-04-17T19:30:03Z INFO __main__ 300: SINGLETONS SOOPManager: /data/stix/SOLSOC/from_soc RidLutManager: file: stixcore/data/publish/rid_lut.csv update: True size: 32748 SPICE: [PosixPath('/data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20240416_002.tm'), PosixPath('/data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20240417_001.tm')] IDBManager: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb Versions: [{'label': '2.26.37', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.37', 'version': ['2', '26', '37']}, {'label': '2.26.36', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.36', 'version': ['2', '26', '36']}, {'label': '2.26.32', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.32', 'version': ['2', '26', '32']}, {'label': '2.26.34', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.34', 'version': ['2', '26', '34']}, {'label': '2.26.38', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.38', 'version': ['2', '26', '38']}, {'label': '2.26.33', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.33', 'version': ['2', '26', '33']}, {'label': '2.26.31', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.31', 'version': ['2', '26', '31']}, {'label': '2.26.35', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.35', 'version': ['2', '26', '35']}] Force version: None History: IntervalTree([Interval(0.0, 640198039.0, '2.26.32'), Interval(640198039.0, 640265396.0, '2.26.31'), Interval(640265396.0, 642159993.0, '2.26.32'), Interval(642159993.0, 643123725.0, '2.26.33'), Interval(643123725.0, 677774249.0, '2.26.34'), Interval(677774249.0, 692380776.0, '2.26.35'), Interval(692380776.0, 761270182.0, '2.26.36'), Interval(761270182.0, 4294967295.0, '2.26.38')]) 2024-04-17T19:30:03Z INFO stixcore.io.RidLutManager 154: download publish lut file: https://datacenter.stix.i4ds.net/api/bsd/info/20240411/20240511 2024-04-17T19:30:03Z INFO stixcore.io.RidLutManager 163: found 1 entries 2024-04-17T19:30:04Z INFO stixcore.io.RidLutManager 172: write total 32748 entries to local storage 2024-04-17T19:30:05Z INFO __main__ 506: Created file: /data/stix/SOLSOC/from_edds/tm/processing/.TM_hourly_req_BatchRequest.PktTmRaw.SOL.0.2024.106.13.44.29.303.tDnr@2024.108.17.00.03.753.1.xml.o0F7l4 2024-04-17T19:30:05Z INFO __main__ 518: Modified directory: /data/stix/SOLSOC/from_edds/tm/processing 2024-04-17T19:30:05Z INFO __main__ 518: Modified file: /data/stix/SOLSOC/from_edds/tm/processing/.TM_hourly_req_BatchRequest.PktTmRaw.SOL.0.2024.106.13.44.29.303.tDnr@2024.108.17.00.03.753.1.xml.o0F7l4 2024-04-17T19:30:05Z INFO __main__ 518: Modified file: /data/stix/SOLSOC/from_edds/tm/processing/.TM_hourly_req_BatchRequest.PktTmRaw.SOL.0.2024.106.13.44.29.303.tDnr@2024.108.17.00.03.753.1.xml.o0F7l4 2024-04-17T19:30:05Z INFO __main__ 518: Modified file: /data/stix/SOLSOC/from_edds/tm/processing/.TM_hourly_req_BatchRequest.PktTmRaw.SOL.0.2024.106.13.44.29.303.tDnr@2024.108.17.00.03.753.1.xml.o0F7l4 2024-04-17T19:30:05Z INFO __main__ 518: Modified file: /data/stix/SOLSOC/from_edds/tm/processing/.TM_hourly_req_BatchRequest.PktTmRaw.SOL.0.2024.106.13.44.29.303.tDnr@2024.108.17.00.03.753.1.xml.o0F7l4 2024-04-17T19:30:05Z INFO __main__ 518: Modified file: /data/stix/SOLSOC/from_edds/tm/processing/.TM_hourly_req_BatchRequest.PktTmRaw.SOL.0.2024.106.13.44.29.303.tDnr@2024.108.17.00.03.753.1.xml.o0F7l4 2024-04-17T19:30:05Z INFO __main__ 518: Modified file: /data/stix/SOLSOC/from_edds/tm/processing/.TM_hourly_req_BatchRequest.PktTmRaw.SOL.0.2024.106.13.44.29.303.tDnr@2024.108.17.00.03.753.1.xml.o0F7l4 2024-04-17T19:30:05Z INFO __main__ 518: Modified file: /data/stix/SOLSOC/from_edds/tm/processing/.TM_hourly_req_BatchRequest.PktTmRaw.SOL.0.2024.106.13.44.29.303.tDnr@2024.108.17.00.03.753.1.xml.o0F7l4 2024-04-17T19:30:05Z INFO __main__ 518: Modified file: /data/stix/SOLSOC/from_edds/tm/processing/.TM_hourly_req_BatchRequest.PktTmRaw.SOL.0.2024.106.13.44.29.303.tDnr@2024.108.17.00.03.753.1.xml.o0F7l4 2024-04-17T19:30:05Z INFO __main__ 518: Modified directory: /data/stix/SOLSOC/from_edds/tm/processing 2024-04-17T19:30:05Z INFO __main__ 518: Modified file: /data/stix/SOLSOC/from_edds/tm/processing/.TM_hourly_req_BatchRequest.PktTmRaw.SOL.0.2024.106.13.44.29.303.tDnr@2024.108.17.00.03.753.1.xml.o0F7l4 2024-04-17T19:30:05Z INFO __main__ 499: Moved file: from /data/stix/SOLSOC/from_edds/tm/processing/.TM_hourly_req_BatchRequest.PktTmRaw.SOL.0.2024.106.13.44.29.303.tDnr@2024.108.17.00.03.753.1.xml.o0F7l4 to /data/stix/SOLSOC/from_edds/tm/processing/TM_hourly_req_BatchRequest.PktTmRaw.SOL.0.2024.106.13.44.29.303.tDnr@2024.108.17.00.03.753.1.xml 2024-04-17T19:30:05Z INFO __main__ 518: Modified directory: /data/stix/SOLSOC/from_edds/tm/processing 2024-04-17T19:30:05Z INFO stixcore.processing.TMTCtoLB 30: Processing file: SOCPacketFile('/data/stix/SOLSOC/from_edds/tm/processing/TM_hourly_req_BatchRequest.PktTmRaw.SOL.0.2024.106.13.44.29.303.tDnr@2024.108.17.00.03.753.2.xml') 2024-04-17T19:30:05Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2403208181-60863.fits 2024-04-17T19:30:05Z INFO __main__ 223: generated LB files: {PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2403208181-60863.fits')} 2024-04-17T19:30:06Z INFO stixcore.idb.idb 522: IDB loaded from /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.38/idb.sqlite 2024-04-17T19:30:06Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20240416_002.tm 2024-04-17T19:30:06Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20240417_001.tm 2024-04-17T19:30:06Z INFO stixcore.io.RidLutManager 174: read rid-lut from stixcore/data/publish/rid_lut.csv 2024-04-17T19:30:07Z WARNING stixcore.products.levelb.binary 322: Incomplete sequence sequence_count sequence_flag scet_coarse -------------- ------------- ----------- 12339 0 766687183 12340 0 766687183 12341 0 766687184 12342 0 766687184 12343 0 766687185 12344 0 766687185 12345 0 766687186 12346 0 766687186 12347 0 766687187 12348 0 766687187 12349 0 766687188 12350 0 766687188 12351 0 766687189 12352 0 766687189 12353 0 766687195 12354 0 766687196 12355 0 766687197 12356 0 766687197 12357 0 766687198 12358 0 766687198 12359 0 766687199 12360 0 766687199 12361 0 766687200 12362 0 766687200 12363 0 766687201 12364 0 766687201 12365 0 766687202 12366 0 766687202 12367 0 766687203 12368 0 766687203 12369 0 766687203 12370 0 766687204 12371 0 766687204 12372 0 766687205 12373 0 766687205 12374 0 766687206 12375 0 766687206 12376 0 766687207 12377 0 766687207 12378 0 766687208 12379 0 766687208 12380 0 766687209 12381 0 766687209 12382 0 766687210 12383 0 766687210 12384 0 766687211 12385 0 766687211 12386 0 766687212 12387 0 766687212 12388 0 766687213 12389 0 766687213 12390 0 766687214 12391 0 766687214 12392 0 766687215 12393 0 766687215 12394 0 766687215 12395 0 766687216 12396 0 766687216 12397 0 766687217 12398 0 766687218 12399 0 766687218 12400 0 766687218 12401 0 766687219 12402 0 766687219 12403 0 766687220 12404 0 766687220 12405 0 766687224 12406 0 766687224 12407 0 766687225 12408 0 766687225 12409 0 766687226 12410 0 766687226 12411 0 766687227 12412 0 766687227 12413 0 766687228 12414 0 766687228 12415 0 766687229 12416 0 766687229 12417 0 766687230 12418 0 766687230 12419 0 766687231 12420 0 766687231 12421 0 766687232 12422 0 766687232 12423 0 766687233 12424 0 766687233 12425 0 766687234 12426 0 766687234 12427 0 766687235 12428 0 766687235 12429 0 766687236 12430 0 766687236 12431 0 766687236 12432 0 766687237 12433 0 766687238 12434 0 766687244 12435 0 766687244 12436 0 766687245 12437 0 766687245 12438 0 766687246 12439 0 766687246 12440 0 766687247 12441 0 766687247 12442 0 766687248 12443 0 766687248 12444 0 766687249 12445 0 766687249 12446 0 766687250 12447 0 766687250 12448 0 766687251 12449 0 766687251 12450 0 766687252 12451 0 766687252 12452 0 766687253 12453 0 766687253 12454 0 766687254 12455 0 766687254 12456 0 766687255 12457 0 766687255 12458 0 766687255 12459 0 766687256 12460 0 766687256 12461 0 766687257 12462 0 766687257 12463 0 766687258 12464 0 766687258 12465 0 766687259 12466 0 766687259 12467 0 766687260 12468 0 766687260 12469 0 766687261 12470 0 766687261 12471 0 766687262 12472 0 766687262 12473 0 766687263 12474 0 766687263 12475 0 766687263 12476 0 766687264 12477 0 766687264 12478 0 766687265 12479 0 766687265 12480 0 766687266 12481 0 766687266 12482 0 766687267 12483 0 766687267 12484 0 766687267 12485 0 766687268 12486 0 766687268 12487 0 766687269 12488 0 766687270 12489 0 766687270 12490 0 766687270 12491 0 766687271 12492 0 766687271 12493 0 766687272 12494 0 766687272 12495 0 766687273 12496 0 766687273 12497 0 766687274 12498 0 766687274 12499 0 766687274 12500 0 766687275 12501 0 766687275 12502 0 766687276 12503 0 766687276 12504 0 766687277 12505 0 766687277 12506 0 766687278 12507 0 766687278 12508 0 766687279 12509 0 766687279 12510 0 766687280 12511 0 766687280 12512 0 766687281 12513 0 766687281 12514 0 766687282 12515 0 766687282 12516 0 766687283 12517 0 766687284 12518 0 766687287 12519 0 766687287 12520 0 766687293 12521 0 766687294 12522 0 766687294 12523 0 766687294 12524 0 766687295 12525 0 766687295 12526 0 766687295 12527 0 766687296 12528 0 766687296 12529 0 766687297 12530 0 766687297 12531 0 766687298 12532 0 766687298 12533 0 766687299 12534 0 766687299 12535 0 766687300 12536 0 766687300 12537 0 766687301 12538 0 766687301 12539 0 766687302 12540 0 766687302 12541 0 766687303 12542 0 766687303 12543 0 766687303 12544 0 766687304 12545 0 766687304 12546 0 766687305 12547 0 766687305 12548 0 766687306 12549 0 766687306 12550 0 766687306 12551 0 766687307 12552 0 766687307 12553 0 766687308 12554 0 766687308 12555 0 766687309 12556 0 766687309 12557 0 766687310 12558 0 766687310 12559 0 766687310 12560 0 766687311 12561 0 766687311 12562 0 766687312 12563 0 766687312 12564 0 766687313 12565 0 766687313 12566 0 766687314 12567 0 766687314 12568 0 766687315 12569 0 766687315 12570 0 766687316 12571 0 766687316 12572 0 766687317 12573 0 766687317 12574 0 766687318 2024-04-17T19:30:07Z INFO __main__ 227: generated L0 files: [] 2024-04-17T19:30:07Z INFO __main__ 231: generated L1 files: [] 2024-04-17T19:30:07Z INFO __main__ 235: generated L2 files: []