Package ghidra.features.bsim.query.client

# Class AbstractSQLFunctionDatabase<VF extends LSHVectorFactory>

java.lang.Object
ghidra.features.bsim.query.client.AbstractSQLFunctionDatabase<VF>

Type Parameters:
  `VF` - `vector factory` implementation class

All Implemented Interfaces:
  `FunctionDatabase`, `SQLFunctionDatabase`, `AutoCloseable`

Direct Known Subclasses:
  `H2FileFunctionDatabase`, `PostgresFunctionDatabase`

public abstract class AbstractSQLFunctionDatabase<VF extends LSHVectorFactory> extends Object implements SQLFunctionDatabase
Defines the BSim `FunctionDatabase` backed by an SQL database. Simple, one-column tables that only contain string data use the `SQLStringTable` class and are defined in this class. More complex tables are defined in their own classes in the `ghidra.features.bsim.query.client.tables` package.

- ## Nested Class Summary

  ## Nested classes/interfaces inherited from interface ghidra.features.bsim.query.FunctionDatabase

  `FunctionDatabase.BSimError, FunctionDatabase.ConnectionType, FunctionDatabase.DatabaseNonFatalException, FunctionDatabase.ErrorCategory, FunctionDatabase.Status`

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `protected final BSimJDBCDataSource`
  `ds`
  `static final String`
  `JAVA_TIME_FORMAT`
  `static final String`
  `SQL_TIME_FORMAT`
  `final int`
  `supportedLayoutVersion`
  `protected final VF`
  `vectorFactory`

- ## Constructor Summary

  Constructors
  Modifier
  Constructor
  Description
  `protected`
  `AbstractSQLFunctionDatabase(BSimJDBCDataSource ds, VF vectorFactory, int supportedLayoutVersion)`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `static void`
  `appendEscapedLiteral(StringBuilder buf, String str)`
  `protected Connection`
  `beginTransaction(boolean lockTablesForWrite)`
  Start a transaction which will not be auto-committed.
  `void`
  `close()`
  Close down (the connection with) the database
  `protected void`
  `closing(Connection c)`
  Perform any required cleanup prior to connection close.
  `int`
  `compareLayout()`
  Return -1 if info layout version is earlier than current client expectation Return 1 if info layout version is later than current client expectation Return 0 if info version and client version are the same
  `protected void`
  `convertDescriptionRows(SimilarityResult simres, List<DescriptionTable.DescriptionRow> descvec, VectorResult vecres, DescriptionManager res, SignatureRecord srec)`
  Convert a low-level list of function rows into full FunctionDescription objects
  `protected void`
  `createDatabase(Configuration config)`
  Generate tables and initialize state for a new database.
  `protected abstract int`
  `deleteVectors(long id, int countdiff)`
  Low level count decrement of a vector record from vectable, if count reaches zero, the record is deleted
  `protected QueryResponseRecord`
  `doQuery(BSimQuery<?> query, Connection c)`
  `protected abstract void`
  `dropDatabase()`
  Drop this database
  `protected void`
  `endTransaction(boolean commit)`
  End a transaction which was started with `beginTransaction(boolean)` and return DB connection to an auto-commit state.
  `protected abstract void`
  `generateRawDatabase()`
  Generate new empty database which corresponds to `ds`.
  `FunctionDatabase.ConnectionType`
  `getConnectionType()`
  `List<String>`
  `getFunctionTags()`
  Returns a list of all function tags in the database.
  `DatabaseInformation`
  `getInfo()`
  `FunctionDatabase.BSimError`
  `getLastError()`
  If the last query failed to produce a response, use this method to recover the error message
  `LSHVectorFactory`
  `getLSHVectorFactory()`
  `BSimServerInfo`
  `getServerInfo()`
  Return the `server info object` for this database
  `FunctionDatabase.Status`
  `getStatus()`
  `String`
  `getURLString()`
  `String`
  `getUserName()`
  `protected Connection`
  `initConnection()`
  Establish PostgreSQL DB pooled `Connection` for this instance.
  `boolean`
  `initialize()`
  Initialize (a connection with) the database.
  `protected void`
  `initializeDatabase(Configuration config)`
  Initialize table state and `config` from existing database.
  `protected void`
  `lockTablesForWrite()`
  Lock appropriate tables during complex write/update operation
  `QueryResponseRecord`
  `query(BSimQuery<?> query)`
  Send a query to the database.
  `protected int`
  `queryFunctions(QueryNearest query, BSimSqlClause filter, ResponseNearest response, DescriptionManager descMgr, Iterator<FunctionDescription> iter)`
  `protected abstract void`
  `queryNearestVector(QueryNearestVector query)`
  Perform nearest vector query based upon specified query which contains response record which should be filled-in with search results.
  `protected abstract int`
  `queryNearestVector(List<VectorResult> resultset, LSHVector vec, double simthresh, double sigthresh, int max)`
  `protected abstract VectorResult`
  `queryVectorId(long id)`
  `protected int`
  `retrieveFuncDescFromVectors(VectorResult dresult, DescriptionManager res, int count, QueryNearest query, BSimSqlClause filter, SimilarityResult simres)`
  Generates `FunctionDescription` objects for each function with signature vector `dresult`.
  `protected void`
  `setConnectionOnTables(Connection db)`
  `protected abstract long`
  `storeSignatureRecord(SignatureRecord sigrec)`
  Make sure the vector corresponding to the SignatureRecord is inserted into the vectable

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

  ### Methods inherited from interface ghidra.features.bsim.query.FunctionDatabase

  `changePassword, getOverviewFunctionsPerStage, getQueriedFunctionsPerStage, isPasswordChangeAllowed`

  ### Methods inherited from interface ghidra.features.bsim.query.SQLFunctionDatabase

  `formatBitAndSQL`

- ## Field Details

  - ### SQL_TIME_FORMAT

    public static final String SQL_TIME_FORMAT
    See Also:
      - Constant Field Values

  - ### JAVA_TIME_FORMAT

    public static final String JAVA_TIME_FORMAT
    See Also:
      - Constant Field Values

  - ### ds

    protected final BSimJDBCDataSource ds

  - ### supportedLayoutVersion

    public final int supportedLayoutVersion

  - ### vectorFactory

    protected final VF extends LSHVectorFactory vectorFactory

- ## Constructor Details

  - ### AbstractSQLFunctionDatabase

    protected AbstractSQLFunctionDatabase(BSimJDBCDataSource ds, VF vectorFactory, int supportedLayoutVersion)

- ## Method Details

  - ### close

    public void close()
    Description copied from interface: `FunctionDatabase`
    Close down (the connection with) the database
    Specified by:
      `close` in interface `AutoCloseable`
    Specified by:
      `close` in interface `FunctionDatabase`

  - ### closing

    protected void closing(Connection c)
    Perform any required cleanup prior to connection close. Partial cleanup has already been performed.
    Parameters:
      `c` - database connection

  - ### initConnection

    protected Connection initConnection() throws SQLException
    Establish PostgreSQL DB pooled `Connection` for this instance.
    Returns:
      database connection
    Throws:
      `SQLException` - if error occurs obtaining connection

  - ### beginTransaction

    protected Connection beginTransaction(boolean lockTablesForWrite) throws SQLException
    Start a transaction which will not be auto-committed. The `endTransaction(boolean)` method must be invoked when the transaction has completed or failed and should be performed within a finally block.
    Parameters:
      `lockTablesForWrite` - if true a table lock will be placed which will clear when `endTransaction(boolean)` is invoked.
    Returns:
      the connection associated with the transaction
    Throws:
      `SQLException` - if an error occurs

  - ### lockTablesForWrite

    protected void lockTablesForWrite() throws SQLException
    Lock appropriate tables during complex write/update operation
    Throws:
      `SQLException` - if an error occurs

  - ### endTransaction

    protected void endTransaction(boolean commit) throws SQLException
    End a transaction which was started with `beginTransaction(boolean)` and return DB connection to an auto-commit state.
    Parameters:
      `commit` - true if transaction should be commited, false to force a rollback.
    Throws:
      `SQLException` - if an error occurs

  - ### initializeDatabase

    protected void initializeDatabase(Configuration config) throws SQLException
    Initialize table state and `config` from existing database. NOTE: it is left to specific implementation to override this method to properly initialize default `config.weightfactory` and `config.idflookup` .
    Parameters:
      `config` - the database configuration
    Throws:
      `SQLException` - if there is a problem opening the database connection

  - ### generateRawDatabase

    protected abstract void generateRawDatabase() throws SQLException
    Generate new empty database which corresponds to `ds`. A new connection may be used to the default database so that a new database may be created.
    Throws:
      `SQLException` - if an error occurs while creating the database

  - ### createDatabase

    protected void createDatabase(Configuration config) throws SQLException
    Generate tables and initialize state for a new database.
    Parameters:
      `config` - the database configuration
    Throws:
      `SQLException` - if there is aproblem opening a connectino to the database

  - ### dropDatabase

    protected abstract void dropDatabase() throws SQLException
    Drop this database
    Throws:
      `SQLException` - if a database error occured

  - ### setConnectionOnTables

    protected void setConnectionOnTables(Connection db)

  - ### appendEscapedLiteral

    public static void appendEscapedLiteral(StringBuilder buf, String str) throws SQLException
    Parameters:
      `buf` - the string builder object
      `str` - the string to parse
    Throws:
      `SQLException` - if there is a zero byte in the string

  - ### convertDescriptionRows

    protected void convertDescriptionRows(SimilarityResult simres, List<DescriptionTable.DescriptionRow> descvec, VectorResult vecres, DescriptionManager res, SignatureRecord srec) throws SQLException, LSHException
    Convert a low-level list of function rows into full FunctionDescription objects
    Parameters:
      `simres` - (optional -- may be null) generate a SimilarityNote for every function
      `descvec` - is the list of low-level function rows
      `vecres` - (optional -- may be null) vector result producing these functions
      `res` - is the DescriptionManager holding the newly generated FunctionDescriptions
      `srec` - (optional -- may be null) is a description object of the vector
    Throws:
      `SQLException` - is there is an error querying tables
      `LSHException` - for internal consistency errors

  - ### deleteVectors

    protected abstract int deleteVectors(long id, int countdiff) throws SQLException
    Low level count decrement of a vector record from vectable, if count reaches zero, the record is deleted
    Parameters:
      `id` - vector row ID
      `countdiff` - the amount to subtract from count
    Returns:
      0 if decrement short of 0, return 1 if record was removed, return -1 if there was a problem
    Throws:
      `SQLException` - if there is a problem creating or executing the query

  - ### queryVectorId

    protected abstract VectorResult queryVectorId(long id) throws SQLException
    Parameters:
      `id` - the vector id
    Returns:
      the vector result
    Throws:
      `SQLException` - if there is a problem creating or executing the query

  - ### getStatus

    public FunctionDatabase.Status getStatus()
    Specified by:
      `getStatus` in interface `FunctionDatabase`
    Returns:
      the status of the current connection with this database

  - ### getConnectionType

    public FunctionDatabase.ConnectionType getConnectionType()
    Specified by:
      `getConnectionType` in interface `FunctionDatabase`
    Returns:
      the type of connection

  - ### getUserName

    public String getUserName()
    Specified by:
      `getUserName` in interface `FunctionDatabase`
    Returns:
      username (being used to establish connection)

  - ### getLSHVectorFactory

    public LSHVectorFactory getLSHVectorFactory()
    Specified by:
      `getLSHVectorFactory` in interface `FunctionDatabase`
    Returns:
      factory the database is using to create LSHVector objects

  - ### getInfo

    public DatabaseInformation getInfo()
    Specified by:
      `getInfo` in interface `FunctionDatabase`
    Returns:
      an information object giving general characteristics and descriptions of this database

  - ### compareLayout

    public int compareLayout()
    Description copied from interface: `FunctionDatabase`
    Return -1 if info layout version is earlier than current client expectation Return 1 if info layout version is later than current client expectation Return 0 if info version and client version are the same
    Specified by:
      `compareLayout` in interface `FunctionDatabase`
    Returns:
      comparison of actual database layout with layout expected by client

  - ### getServerInfo

    public BSimServerInfo getServerInfo()
    Description copied from interface: `FunctionDatabase`
    Return the `server info object` for this database
    Specified by:
      `getServerInfo` in interface `FunctionDatabase`
    Returns:
      the server info object

  - ### getURLString

    public String getURLString()
    Specified by:
      `getURLString` in interface `FunctionDatabase`

  - ### getLastError

    public FunctionDatabase.BSimError getLastError()
    Description copied from interface: `FunctionDatabase`
    If the last query failed to produce a response, use this method to recover the error message
    Specified by:
      `getLastError` in interface `FunctionDatabase`
    Returns:
      a String describing the error

  - ### initialize

    public boolean initialize()
    Description copied from interface: `FunctionDatabase`
    Initialize (a connection with) the database. If initialization is not successful, this routine will return false and an error description can be obtained using getLastError
    Specified by:
      `initialize` in interface `FunctionDatabase`
    Returns:
      true if the database ready for querying

  - ### storeSignatureRecord

    protected abstract long storeSignatureRecord(SignatureRecord sigrec) throws SQLException
    Make sure the vector corresponding to the SignatureRecord is inserted into the vectable
    Parameters:
      `sigrec` - is the SignatureRecord
    Returns:
      the computed id of the vector
    Throws:
      `SQLException` - if there is a problem creating or executing the query

  - ### retrieveFuncDescFromVectors

    protected int retrieveFuncDescFromVectors(VectorResult dresult, DescriptionManager res, int count, QueryNearest query, BSimSqlClause filter, SimilarityResult simres) throws SQLException, LSHException
    Generates `FunctionDescription` objects for each function with signature vector `dresult`. If a non-null `filter` is provided, `FunctionDescription`s are only created for functions with pass the filter.
    Parameters:
      `dresult` - `VectorResult` representing a matching vector
      `res` - `DescriptionManager` container for the results
      `count` - number of function descriptions which have already been returned for this base vector
      `query` - contains various thresholds for the query
      `filter` - specifies additional conditions functions (and exes) must meet after meeting sim/signif threshold
      `simres` - receives the list of results and their similarity to the base vector
    Returns:
      number of
    Throws:
      `SQLException` - if there is an error issuing the query
      `LSHException` - if executable record creation fails

  - ### queryNearestVector

    protected abstract int queryNearestVector(List<VectorResult> resultset, LSHVector vec, double simthresh, double sigthresh, int max) throws SQLException
    Parameters:
      `resultset` - the list of result set objects to populate
      `vec` - the vector containing the saveSQL query statement
      `simthresh` - the similarity threshold
      `sigthresh` - the confidence threshold
      `max` - the max number of results to return
    Returns:
      the number of results returned
    Throws:
      `SQLException` - if there is a problem creating or executing the query

  - ### queryNearestVector

    protected abstract void queryNearestVector(QueryNearestVector query) throws SQLException
    Perform nearest vector query based upon specified query which contains response record which should be filled-in with search results.
    Parameters:
      `query` - the nearest vector query to perform
    Throws:
      `SQLException` - if there is an error issuing the query

  - ### doQuery

    protected QueryResponseRecord doQuery(BSimQuery<?> query, Connection c) throws LSHException, SQLException, FunctionDatabase.DatabaseNonFatalException
    Throws:
      `LSHException`
      `SQLException`
      `FunctionDatabase.DatabaseNonFatalException`

  - ### query

    public QueryResponseRecord query(BSimQuery<?> query)
    Description copied from interface: `FunctionDatabase`
    Send a query to the database. The response is returned as a QueryResponseRecord. If this is null, an error has occurred and an error message can be obtained from getLastError
    Specified by:
      `query` in interface `FunctionDatabase`
    Parameters:
      `query` - an object describing the query
    Returns:
      the response object or null if there is an error

  - ### queryFunctions

    protected int queryFunctions(QueryNearest query, BSimSqlClause filter, ResponseNearest response, DescriptionManager descMgr, Iterator<FunctionDescription> iter) throws SQLException, LSHException
    Parameters:
      `query` - the query to execute
      `filter` - the function filter
      `response` - the response object
      `descMgr` - the executable descriptor
      `iter` - the function iterator
    Returns:
      the number of unique results found
    Throws:
      `SQLException` - if there is an error issuing the query
      `LSHException` - if there is a problem creating executable records

  - ### getFunctionTags

    public List<String> getFunctionTags()
    Returns a list of all function tags in the database.
    Returns:
      list of function tags