Perform Semantic Find

Performs a semantic find in a target field for the specified text, image, or embedding vectors.

See also 

Options 

  • Query by specifies whether the query is Natural language (text), embedding Vector data, or an Image (container data).

  • Record set selects whether to search All records or Current found set.

  • Target field specifies a text or container field to search in. The contents of the field must be embedding vectors, either as a JSON array if a text field, or as binary data if a container field.

  • Return count is a numeric expression for the number of semantically similar records to return as the found set. If not specified, the default is a maximum of 10 records.

  • Cosine similarity condition specifies how the cosine similarity of the data is compared to the Cosine similarity value. If not specified, the found set isn't limited by a cosine similarity condition.

  • Cosine similarity value is the threshold used along with the Cosine similarity condition to determine whether data is semantically similar or dissimilar. This numeric expression must have a value between -1 (dissimilar) and 1 (similar), inclusive.

  • Save result saves the found set in a text field or variable as a JSON array of record IDs and their cosine similarity values sorted in order of descending cosine similarity.

Options available only when Query by is Natural language or Image:

  • Account Name is a text expression for the name of the AI account to use. In the current file, use the Configure AI Account script step to set up the account and assign it this name any time before this script step runs.

  • Embedding Model is the name of the model to generate embedding vectors. Specify the model name as a text expression. For supported models, see FileMaker technical specifications.

Options available only when Query by is Natural language:

  • Text is a text expression for the natural language text query. The resulting text is sent to the model for embedding vectors, which are then compared with the embedding vectors in Target field for similarity.

Options available only when Query by is Vector data:

  • Vector is the embedding vectors that represent the text or images to find. This is an expression that returns container data, such as a reference to a container field. Results will be semantically similar to the text or image this vector data represents.

Options available only when Query by is Image:

  • Image is any expression that returns container data representing the image to find.

Compatibility 

Product Supported
FileMaker Pro Yes
FileMaker Go Yes
FileMaker WebDirect Yes
FileMaker Server Yes
FileMaker Cloud Yes
FileMaker Data API Yes
Custom Web Publishing Yes

Originated in version 

21.0

Description 

This script step looks for records in which the content of the Target field is semantically similar to the query. When searching for text, this can provide more useful results than looking for literal keyword matches as the Perform Find does.

When searching for images, you can specify natural language text or text embedding vectors that describe the images you're searching for. Or you can specify an image (or embedding vectors for an image) that's similar to the images you're searching for.

This script step has these modes of operation determined by the Query by option:

  • When set to Natural language, you provide the query as natural language text in the Text option. This script step sends that query text to the specified model for conversion to text embedding vectors.

  • When set to Image, you provide the query as container data for an image in the Image option. This script step sends that query image to the specified model for conversion to image embedding vectors.

  • When set to Vector data, you provide the embedding vectors as container data for the query in the Vector option. If you commonly use certain queries, it may be more efficient to get embedding vectors for those queries once and store them rather than frequently get them from the model.

    You must use the same embedding model to generate the vectors for the Vector option as you do for the embedding vectors in the Target field.

For image as well as text embedding vectors, you can use the models supported by the AI Model Server installed with FileMaker Server. See the Configuring AI services in FileMaker Server Help.

Starting with either all records in the current table or only the records in the current found set (set by the Record set option), this script step compares the data in Target field with the query for semantic similarity (or dissimilarity). The result is a constrained (narrowed) found set of no more than Return count records that meet the cosine similarity conditions. The resulting found set is sorted in order of descending cosine similarity so the most relevant records are first.

Notes 

  • Normalized embedding vectors are required. All embedding vectors must be generated from the same model to ensure compatibility and performance; mixing embedding vectors from different models isn't supported.

Example 1 

Configures an AI account, goes to the Meeting Details layout, then in all records, performs a semantic find for Text (as natural language) in the Meetings::Note_Embedding container field, which contains embedding vectors for the contents of the Meetings::Note field (see Example 1 in Insert Embedding in Found Set). For more useful results, the return count is limited to 10 and the cosine similarity between Text and the contents of Meetings::Note_Embedding field must be greater than 0.4. The resulting found set is sorted with the most relevant records first.

Copy 
Configure AI Account [ Account Name: "my-account" ; Model Provider: OpenAI ; API key: "sk-RZCtpWT..." ]

Go to Layout [ "Meeting Details" (Meetings) ; Animation: None ]

Perform Semantic Find [ Query by: Natural language ; Account Name: "my-account" ; Embedding Model: "text-embedding-3-small" ; Text: "Recruitment, job definition, training plan" ; Record set: All records ; Target field: Meetings::Note_Embedding ; Return count: 10 ; Cosine similarity condition: greater than ; Cosine similarity value: .4 ]

Example 2

Configures an AI account, goes to the Meeting Details layout, then asks the user for text to find. The GetEmbedding function sends that text to the model, then Set Field stores the embedding vectors in the global container field Meetings::Input. Assuming the Meetings::Note_Embedding container field contains embedding vectors for the Meetings::Note field's contents (see Example 1 in Insert Embedding in Found Set), Perform Semantic Find compares the vector data in Meetings::Input with Meetings::Note_Embedding in all records and returns a found set that meets the return count and similarity criteria.

Copy 
Configure AI Account [ Account Name: "my-account" ; Model Provider: OpenAI ; API key: "sk-RZCtpWT..." ]

Go to Layout [ "Meeting Details" (Meetings) ; Animation: None ]
Show Custom Dialog [ "Find meetings with notes about:" ; $Input ]

Set Field [ Meetings::Input ; GetEmbedding ( "my-account" ; "text-embedding-3-small" ; $Input ) ]

Perform Semantic Find [ Query by: Vector data ; Meetings::Input ; Record set: All records ; Target field: Meetings::Note_Embedding ; Return count: 10 ; Cosine similarity condition: greater than ; Cosine similarity value: .5 ; Save result: $$result]

With the Save result option selected, for a given query that returns three records, JSONFormatElement($$result) returns:

Copy 
[
    {
        "recordId" : "7",
        "similarity" : 0.999999947111836
    },
    {
        "recordId" : "1",
        "similarity" : 0.529809641529481
    },
    {
        "recordId" : "5",
        "similarity" : 0.524595621039394
    }
]

Example 3 

Searches a set of images for those that best match a text description. The script configures an AI account for the AI Model Server installed with FileMaker Server, goes to the Meeting Details layout, shows all records, then gets embedding vectors for the images in the Meetings::Image field using an image embedding model and stores them as binary data in the Meetings::Image_Embedding container field. Perform Semantic Find sends a natural language query text ("People sitting around a table") to a text embedding model using the same account, then compares the result with Meetings::Image_Embedding in all records and returns a found set that meets the similarity criteria.

Copy 
Configure AI Account [ Account Name: "my-account" ; Model Provider: Custom ; Endpoint: "https://my-server.example.com:8080/" ; Verify SSL Certificates ; API key: Global::API_Key ]

Go to Layout [ "Meeting Details" (Meetings) ; Animation: None ]
Show All Records

Insert Embedding in Found Set [ Account Name: "my-account" ; Embedding Model: "clip-ViT-B-32" ; Source Field: Meetings::Image ; Target Field: Meetings::Image_Embedding ]

Perform Semantic Find [ Query by: Natural language ; Account Name: "my-account" ; Embedding Model: "sentence-transformers/clip-ViT-B-32-multilingual-v1" ; Text: "People sitting around a table" ; Record set: All records ; Target field: Meetings::Image_Embedding ; Cosine similarity condition: greater than ; Cosine similarity value: .7 ]

Example 4 

Searches a set of images for those that are most similar to the specified image. The script makes the same preparations as in the previous example, but in this example, Perform Semantic Find queries by the image in the Global::Temp_Image container field. Notice that the embedding model used for the query image is the same as the one that created embedding vectors for the images being searched. The script returns a found set that meets the similarity criteria.

Copy 
Configure AI Account [ Account Name: "my-account" ; Model Provider: Custom ; Endpoint: "https://my-server.example.com:8080/" ; Verify SSL Certificates ; API key: Global::API_Key ]

Go to Layout [ "Meeting Details" (Meetings) ; Animation: None ]
Show All Records

Insert Embedding in Found Set [ Account Name: "my-account" ; Embedding Model: "clip-ViT-B-32" ; Source Field: Meetings::Image ; Target Field: Meetings::Image_Embedding ]

Perform Semantic Find [ Query by: Image ; Account Name: "my-account" ; Embedding Model: "clip-ViT-B-32" ; Image: Global::Temp_Image ; Record set: All records ; Target field: Meetings::Image_Embedding ; Cosine similarity condition: greater than ; Cosine similarity value: .7 ]

Related topics