Popular Models

The FinBERT model is a sentiment analysis model. The following use cases explain how you might utilize it in trading algorithms:

• Analyze the sentiment of the latest news articles for specific companies, then form a long-short portfolio with assets that have the most postive and most negative news sentiment.
• Monitor the sentiment of regulatory alerts in a risk management model and liquidate holdings when sentiment is extremely negative.
• Generate sentences based on information from other datasets and then feed them into the model to determine the sentiment. For example, you could use the US Government Contracts dataset to create the string "The Department of State grants AAPL a contract for the purchase of mobile phones".

Follow these steps to load the pre-trained FinBERT model:

1. Import the model and tokenizer classes.

from transformers import TFBertForSequenceClassification, BertTokenizer

2. Define the path were the model is stored.

In QuantConnect Cloud, the path is ProsusAI / finbert.

model_path = "ProsusAI/finbert"

3. Create a TFBertForSequenceClassification model.

self._model = TFBertForSequenceClassification.from_pretrained(model_path, local_files_only=True)

4. Create a BertTokenizer object.

self._tokenizer = BertTokenizer.from_pretrained(model_path, local_files_only=True)

5. (Optional) Set the seed to enable reproducibility.

from transformers import set_seed
set_seed(1, True)

The FinBERT model is pre-trained, so you don't need to fine-tune it. Fine-tuning the model just tailors it to your specific use case. Follow these steps to fine-tune it:

1. Import the Dataset class.

from datasets import Dataset

2. Load the pre-trained model.
3. Compile the model with an optimizer and loss function.

model.compile(
    optimizer=tf.keras.optimizers.Adam(learning_rate=3e-5), 
    loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
)

You only need to compile the model once. TensorFlow offers many other optimizer and loss classes you can use.

4. Create a DataFrame that contains your training samples.

The DataFrame should have two columns, named "text" and "label". The label must be an integer that represents the sentiment class. By default, the FinBERT model has three classes. Class zero represents negative sentiment, class one represents neutral sentiment, and class two represents positive sentiment.

samples = pd.DataFrame(columns=['text', 'label'])
# Add rows to the DataFrame...

5. Convert the samples DataFrame to a tf.data.Dataset object.

dataset = Dataset.from_pandas(samples)

6. Tokenize the text in each training sample.

dataset = dataset.map(
    lambda sample: self._tokenizer(sample['text'], padding='max_length', truncation=True)
)

7. Call the model's prepare_tf_dataset method.

dataset = model.prepare_tf_dataset(dataset, shuffle=True, tokenizer=self._tokenizer)

8. Call the model's fit method.

model.fit(dataset, epochs=2)

Follow these steps to analyze the sentiment of some text with FinBERT:

1. Load the model.
2. (Optional) Fine-tune the model.
3. Get the text you want the model to analyze.

The model can analyze a single sentence or a list of sentences.

content = "AAPL stock price spikes after record-breaking sales figures."

4. Tokenize the text(s).

inputs = self._tokenizer(content, padding=True, truncation=True, return_tensors='tf')

For more information about how to tokenize, see the PreTrainedTokenizer.__call__ reference on the Hugging Face website.

5. Perform dictionary unpacking on the preceding result and pass it to the model as input.

outputs = self._model(**inputs)

6. Apply softmax to the outputs to get probabilities of each sentiment class.

scores = tf.nn.softmax(outputs.logits, axis=-1).numpy()

The result of the preceding operation is a two dimensional numpy array. Each element in the two dimensional array is a list that contains the probability that the sentiment of the corresponding sentence is negative, neutral, or postiive, respectively. For example, you may get the following result if you use a single sentence as input. The result shows that the input is more positive than negative, but is likely neutral.

array([[0.21346861, 0.46771246, 0.318819]])

The following algorithm selects a volatile asset at the beginning of each month. It gets the Tiingo News articles that were released for the asset over the previous 10 days and then feeds them into the pre-trained FinBERT model. It then aggregates the sentiment scores of all the news releases. If the aggregated sentiment score is positive, it enters a long position for the month. If it's negative, it enters a short position for the month.

The following algorithm selects a volatile asset at the beginning of each month. It gets the Tiingo News articles that were released for the asset over the previous 30 days to generate the training set. The label is the market return that occurs from the current news release to the next news release. The algorithm then fine-tunes the model, calculates the sentiment, and rebalances the portfolio.

These algorithms require a GPU node.