Class BlockUploader

java.lang.Object
uk.ac.starlink.ttools.cone.BlockUploader
public class BlockUploader extends Object
Works with an UploadMatcher dividing the input table into chunks and uploading them separately to produce an arbitrarily large result while each upload/match operation is of a limited size.
Since:
15 May 2014
Author:
Mark Taylor

  • Constructor Details

    • BlockUploader

      public BlockUploader(UploadMatcher umatcher, int blocksize, long maxrec, String outName, uk.ac.starlink.table.JoinFixAction uploadFixAct, uk.ac.starlink.table.JoinFixAction remoteFixAct, ServiceFindMode serviceMode, boolean oneToOne, boolean uploadEmpty)
      Constructor.
      Parameters:
      umatcher - upload matcher
      blocksize - maximum number of rows per uploaded block
      maxrec - maximum number of output rows accepted in total
      outName - name of output table
      uploadFixAct - name deduplication policy for upload table
      remoteFixAct - name deduplication policy for remote table
      serviceMode - upload match mode
      oneToOne - true iff output rows match 1:1 with input rows
      uploadEmpty - determines behaviour if there are no input rows: true means attempt the match anyway, false means throw an IOException

  • Method Details

    • runMatch

      public uk.ac.starlink.util.Bi<uk.ac.starlink.table.StarTable, BlockUploader.BlockStats> runMatch(uk.ac.starlink.table.StarTable inTable, QuerySequenceFactory qsFact, uk.ac.starlink.table.StoragePolicy storage) throws IOException
      Performs an upload join in blocks.

      As currently implemented, the input table needs to have random access, and the output table has random access. It would be possible to stream on input and output to some extent, though each input chunk will still have to be random access.

      The row indices given by ConeQueryRowSequence.getIndex calls associated with the supplied qsFact object must correspond to the row indices in the supplied inTable.

      The service may truncate results for some of the blocks. Warnings will be written through the logging system if this happens. Summary information about such truncation will also be returned in the BlockStats object.

      Parameters:
      inTable - input table, must have random access
      qsFact - object to generate positional queries when applied to a table
      storage - storage policy for storing raw result table
      Returns:
      the result table and additional information about the match
      Throws:
      IOException

Copyright © 2026 Central Laboratory of the Research Councils. All Rights Reserved.