Yes, you too can create a search engine with the OpenAI API. And you can put it in a Java application.
How? With the assistance of something called Embeddings.
In this guide, I'll go over the concept of Embeddings. Then I'll walk you through a real-world search engine example in Java.
Keep in mind: this tutorial is part of a series. And if you're brand spankin' new to this OpenAI-with-Java stuff, you might need to start at the beginning.
You can also check out the code on GitHub.
Disclaimer
If you read the title of this article and thought to yourself: "Hey, I can duplicate Google in 5 minutes!" let me disabuse you of that notion right now.
Before you can duplicate Google, you have to index the contents of pretty much the entire world wide web. And that's going to take some time.
And storage.
You'd also burn up your tokens before you even got 1/1,000,000th of the way done.
So no. You won't duplicate Google with this exercise.
However, you can (and will) create a search engine for a much smaller universe of plain-text info. That can be the starting point for your Google clone.
And now on to the guide.
It All Starts With Embeddings
If you want to create a search engine with OpenAI, you're going to need to get Embeddings.
But what the heck are Embeddings?
Well, simply put, they're lists of numbers. Sometimes they're called vectors.
Here's what you do: send some text to the OpenAI API /embeddings endpoint. In response, the API will send you lists of numbers.
Each line (or separate string) in the input gets its own list of numbers. So if you send three lines, you'll get three lists of numbers.
But what's the point of those numbers?
They're used to determine how much separate strings are related. It all comes down to math.
Think about how search engines work. Let's say you go to your favorite search engine and search for "blue zebras." That search engine will return a list of websites with content that most closely resembles "blue zebras." The site that's the closest match should show up at the top of the list.
But how does it determine "the closest match"?
Well, with OpenAI, it uses Embeddings.
For starters, all the content that users can search gets translated to Embeddings.
Then, the actual search query (in this case, "blue zebras") gets translated to Embeddings as well.
Then, finally, the search application uses not-so-basic math to determine which Embeddings in the search universe most closely match the Embeddings derived from "blue zebras."
So it's all about numbers.
But Why Embeddings?
But why translate everything to lists of numbers? Why not just go search for the phrase "blue zebras"?
Ah, but remember we're dealing with AI here. The model is smart enough to know the English language.
And it understands more than just simple words. It understands concepts.
So when you design a search application, you want your users to have confidence that it will return contents that match on concepts and not just on words.
Let me show you a ridiculously simple example. Let's say the entire universe of your data to be searched includes only three lines:
- I like dogs
- I hate cats
- I'm okay with otters
Now let's say somebody searches for "I like dogs." Which bullet point matches?
Well that's a no-brainer. The first one.
But let's say somebody searches for "I love canines." Which bullet point matches?
Well, technically, none of them match perfectly.
But which one is the closest match? Obviously, the first one again.
Embeddings will enable your application to return that first result when people search for "I love canines."
You won't have to code your own thesaurus or any kind of dictionary lookup. OpenAI will take care of all of that for you.
This business of searching based on concepts, by the way, is called a semantic search. It's in contrast to a lexical search that searches for exact words (with variants).
How Do Embeddings Play the Match Game?
So how, exactly, do Embeddings give us the best match?
As I noted before, it all comes down to math. But what kind of math, specifically?
Well this ain't no math class but what you're looking for is something called cosine similarity.
Simply put: two Embeddings are compared using cosine similarity. That comparison returns a number between -1 and 1.
A number of -1 means the two items are total opposites. A number of 1 means they're identical.
Your application will use cosine similarity to compare the Embedding of the search term to the Embeddings of all the content you want to search. Then, you'll just order those results in descending numerical order.
Why descending? Because the results with the highest numbers are the closest matches.
Then, you spit out the results to the UI. It's that simple.
Let's go through an example with code.
An Example With Code
Remember: the code here builds on the basic stuff that you've hopefully already done. If not, you should go to the start of this series and work forward.
First up, let's get a Calculations class going with a method that computes cosine similarity:
public class Calculations {
public static double cosineSimilarity(final List<Double> vectorA, final List<Double> vectorB) {
double dotProduct = 0.0;
double normA = 0.0;
double normB = 0.0;
for (int i = 0; i < vectorA.size(); i++) {
dotProduct += vectorA.get(i) * vectorB.get(i);
normA += Math.pow(vectorA.get(i), 2);
normB += Math.pow(vectorB.get(i), 2);
}
return dotProduct / (Math.sqrt(normA) * Math.sqrt(normB));
}
}Truth be told, I hijacked that code from StackOverflow. But who wouldn't do that?
Anyway, you'll be needing that later.
Next, let's get the text that you want users to search.
I'm taking a page from the OpenAI tutorial playbook and using data from Kaggle. In this case, it's old Amazon food reviews.
The text looks like this:
Title: Good Quality Dog Food; Content: I have bought several of the Vitality canned dog food products and have found them all to be of good quality. The product looks more like a stew than a processed meat and it smells better. My Labrador is finicky and she appreciates this product better than most.
Title: Not as Advertised; Content: Product arrived labeled as Jumbo Salted Peanuts...the peanuts were actually small sized unsalted. Not sure if this was an error or if the vendor intended to represent the product as "Jumbo".
...Each line is separated into two parts:
- The title
- The content
The title and the content are spearated by a semicolon.
The file is saved in a text file called amazon-food-reviews.txt. And it's checked in if you want to see the whole thing.
Why a file? Because the text is far too long for a simple static String assignment in your code. Much better to read it from a file.
Model Behavior
Next, let's get a couple of models going. First up: SearchDataSet:
public class SearchDataSet {
private String inputFile;
private String embeddingsFile;
...
}I left the getters and setters out (as usual).
The whole point of that class is to associate the text input file with the Embeddings input file.
The text input file is that amazon-food-reviews.txt file that you saw in the previous section. But that file just gives you plain text.
You need a separate file to store the Embeddings.
Of course, you could just store the Embeddings in memory. But then you'd need to go out to OpenAI and get those Embeddings for the same text over and over again. You'd blow through your tokens.
And that will cost you more money.
So the rule of thumb here is to get the Embeddings for the text content once. And then persist the Embeddings in a flat file.
In a future article, I'll show you how to persist Embeddings in a database.
Next, get the SearchResult class going:
public class SearchResult implements Comparable<SearchResult> {
private String text;
private Double score;
...
@Override
public int compareTo(final SearchResult o) {
return Double.compare(o.getScore(), score);
}
}That class lives up to its name. It represents a Search Result that combines both the text that matches the query and the cosine similarity score.
The compareTo() method makes it possible to sort the results by score. Note that the code will sort the results in DESCENDING order. That's because you want the matches with the highest scores at the top.
EmbeddingsHelper
Now create an EmbeddingsHelper class that makes it easy to get the Embeddings.
Remember: we're sticking with my main man Theo Kanning's Java API for OpenAI. It works well so I don't see any reason to change horses mid-stream.
Anyway, here's the code for the class:
public class EmbeddingsHelper {
private static Logger LOG = LoggerFactory.getLogger(EmbeddingsHelper.class);
private static final String DEFAULT_MODEL = "text-embedding-ada-002";
private static EmbeddingRequest getEmbeddingRequest(final List<String> inputText) {
final EmbeddingRequest request = EmbeddingRequest
.builder()
.model(DEFAULT_MODEL)
.input(inputText)
.build();
return request;
}
public static List<Embedding> getEmbeddings(final List<String> inputText) {
final OpenAiService service = OpenAiServiceHelper.getOpenAiService();
final EmbeddingRequest request = getEmbeddingRequest(inputText);
final List<Embedding> embeddings = service
.createEmbeddings(request)
.getData();
return embeddings;
}
public static boolean persistEmbeddings(final List<String> inputText, final String pathStr) {
try {
final List<Embedding> embeddings = getEmbeddings(inputText);
saveFile(embeddings, pathStr);
} catch (Exception e) {
LOG.error("Problem saving embeddings!", e);
return false;
}
return true;
}
...
}Note the deafault model at the top. As of this writing, that's the model that our friends at Microsoft recommend you use when retrieving Embeddings. You'll want to double-check that if you're reading this article well after its publication date.
The first method instantiates an object that represents an Embeddings request.
The second method actually sends the request and returns a List of Embedding objects.
The third method persists the Embeddings as numbers in a flat file. Here's a sample of what that looks like:
9.459148E-5,0.0025626542,-0.012473803,-0.0037441377,-0.0047625434,0.026425293,-0.005491402,-0.045368966,-0.033547472,-0.02630548,0.0024178808,0.051945336,-0.02187242,-0.012846553,-0.025533356,0.010869647,0.039910845,0.0070289946,-0.0071022133,-0.006569714,-0.027370479,0.014510614,0.001467702,0.0021899045,-0.0102373045,0.0036908877,0.012580303,-0.012700115,-0.0013029599,0.025812916,0.027663354,0.012247491,-0.020980483,-0.0063999793,0.015149613,-0.0030369116,0.008007462,-0.019715797,0.028541978,-0.021725982,0.0053948862,0.006117089,0.014523926,-0.010922898,-0.016187988,0.0047226055,-0.015855175,0.0031750288,0.003531138,0.003126771,0.041801218,0.0058541675,-0.016227925,-9.698357E-6,0.020434672,-0.0066129793,0.014630426,-0.017625736,0.013605365,-0.021686045,0.001221421,0.008220462,-0.007068932,-0.0030518882,-0.0016391003,-0.028408853,-0.01738611,0.0044397153,0.008020775,0.01058343,0.014989863,0.009751399,-0.003907216,0.031364225,0.014377489,0.009584992,-0.026159042,-0.0014061317,0.01125571,-0.017452674,0.011342241,-0.041561592,-0.005518027,0.033387724,0.009658211,0.004802481,-0.0018654126,0.01331249,-0.0015658815,-0.012613584,0.010410367,0.03615672,0.0124937715,0.010337148,-0.035544347,-0.012087741,0.02432192,0.035863847,-0.014270989,-0.02486773,-0.006200292,-0.015748676,-0.0034213099,-0.006383339,-0.021832483,-0.0071354946,-0.018517673,-0.013751802,0.03373385,-0.005770964,-0.010397054,0.0065064793,0.018530985,-0.026571728,-0.012314053,0.020288235,0.038047094,-0.01850436,-0.015269426,-0.008599868,0.004013716,0.01061671,0.04283959,0.0016182995,0.016720487,0.006383339,-0.0100243045,-0.0031151227,0.0056112143,0.006063839,0.019862235,0.0333611,0.017945236,0.008133931,-0.020554485,0.024401793,-0.021206796,0.0047758557,0.011615147,-0.010137461,0.01935636,0.028781602,-0.014457364,0.004406434,-0.007774494,0.019715797,-0.009451868,0.044516966,-0.015868487,-8.4201497E-4,-0.003997075,-0.028062727,0.031044725,0.007867682,-0.0033547473,-0.006739448,0.0070889005,0.009212242,5.782613E-4,0.022418233,0.0029570367,-0.009764711,-0.034159847,-0.011708335,-0.0042533404,0.028488727,0.036822345,-0.019595984,-0.009711461,0.0018953657,0.02229842,0.032695476,-0.008872774,0.019955423,-0.011482023,0.007541525,0.0065663853,-0.021406483,-0.037301596,7.367631E-4,0.004812465,0.003096818,0.04140184,0.018091673,-6.302632E-4,0.0023230293,0.027769852,0.0030835054,0.030086227,-0.006862588,0.020141797,0.012846553,0.0015375925,-0.02128667,-0.6407035,-0.023456607,-0.008806212,-0.019116735,0.033653975,0.01940961,0.002229842,0.015469112,-0.004978871,0.012646865,-0.019968735,0.019316422,-0.0045561995,0.009584992,-0.0031434116,-0.031577226,0.009638242,-0.0333611,-0.010463617,0.025000855,-0.0063467296,0.0060471985,-0.010610054,-0.03628985,0.015602238,-0.0041401843,0.012307396,-0.0064266045,0.015389238,0.02529373,-0.015868487,0.035863847,-0.0028455446,0.023190357,0.052211583,-1.1128409E-4,0.0063134483,0.021313297,0.030032976,0.027902978,-0.03373385,-0.011601835,0.0063167764,0.005391558,-0.0319766,0.003286521,0.017945236,0.026957791,-0.021486359,-0.015602238,-0.0053416365,-0.014577176,-0.013552114,-0.010164086,0.007075588,0.002664162,0.037674345,-0.026718166,0.014124552,0.0073285257,0.02545348,0.008027432,-0.0045828247,-0.0063966513,-0.030352477,0.015335988,-0.012021178,-0.009012556,0.021260045,-0.020354796,0.0010200695,0.035783973,-0.012973021,0.0038107,-3.4633273E-4,0.005285058,0.015229488,0.011701678,0.014563863,-0.0068126665,0.013671927,-0.015602238,-0.0168403,-0.007801119,-0.013658614,0.010816398,-0.026571728,-0.008539962,0.009525087,-0.016720487,0.0012297413,0.013805051,0.001550905,-0.037834097,-0.03535797,0.008313649,-0.025280418,0.026292168,0.023523169,-0.027157478,-0.033787098,0.027130853,0.014270989,0.0059140734,0.013172708,0.03620997,-0.021845795,0.0012463818,-0.013405677,-0.028808227,0.019436235,-0.009245524,-0.002584287,-0.032642223,-0.04049659,-0.020061921,0.009744742,0.010949522,-0.012899802,-0.018451111,0.0041401843,-0.006263526,0.002283092,6.3234323E-4,-0.02224517,0.0149632385,-0.017039986,0.0092588365,-0.0019286469,-0.018198173,-0.007661338,0.025746355,0.012440521,0.005062074,-0.021339921,0.037780844,0.012487115,-0.0074616503,0.019263173,-0.0030019665,0.0024578185,-0.0068892133,0.0047991527,-0.037993845,0.003754122,-0.012580303,-0.019436235,-0.006270183,-0.012686803,0.010270585,-0.021246733,0.0064898385,-0.018371236,-0.0015750339,0.01550905,0.0031184508,0.0044796527,-0.0110893035,0.018211486,-0.023350107,0.022751044,0.010516867,-0.014124552,0.004729262,-0.00439645,-0.0120677715,0.020248296,0.008639806,-0.018650798,-0.019130047,0.029740103,-0.0149632385,-0.0082803685,0.011408804,0.015029801,6.955776E-4,-0.0304856,0.005308355,0.009591648,0.02219192,-0.0043664966,0.011275679,-0.016187988,-0.018317986,0.016693862,0.009525087,0.015895113,0.060012702,-0.013385708,0.037221722,0.0090458365,0.008766274,0.014617113,-0.0031933335,-0.025972668,-0.018890422,-0.008373556,0.020594422,0.012094397,-0.018730672,0.020847358,-0.028222477,0.019862235,-0.0039271843,0.03658272,-0.014404113,0.0076214,-0.016773736,0.0013686903,0.0147901755,-0.0043332153,0.014018051,-0.016920174,-0.0035777315,0.00957168,0.026065854,-0.003674247,-0.005165246,-0.012007866,0.014670364,0.0041401843,0.004017044,-0.0073817754,-0.012027835,-0.0122807715,0.026438605,-0.009611618,0.0325091,-0.017678985,-0.0029487165,-0.0052451207,0.019702485,0.029393977,0.012380615,0.021539608,-0.02112692,0.015415863,-0.029180977,0.009817961,0.013172708,0.02235167,0.022684483,0.0023845998,0.005677777,0.014976551,0.0020318187,0.019396298,0.010250617,-0.013871614,-0.013232615,-0.0029786695,-0.01582855,-0.034106597,-0.010037617,0.0064831823,-0.027956229,-0.002694115,-0.014870051,0.01893036,0.0049688867,0.014417427,0.0077012754,0.02096717,0.0026242244,0.02513398,0.0057410114,-0.028222477,-0.0013179365,-0.0043531843,-0.0035577628,-0.0043698247,8.3619074E-5,0.0049056523,-0.02171267,0.006929151,0.0318701,-0.021047046,0.021060359,-0.006989057,0.034346223,-0.025533356,-0.0056078862,0.0050121523,0.011062679,0.010357117,0.017705612,-0.028861478,-0.010383742,2.040555E-4,0.017039986,0.013725176,0.019702485,-0.008626494,0.013186021,0.0035111692,0.005195199,0.025679793,-0.021672733,0.022364983,0.01946286,0.016028237,-0.010030961,-0.002201553,-0.012999646,0.026238916,-0.022657856,-0.022231858,0.009817961,-0.0079808375,-0.0172663,0.02213867,-0.0048291055,-0.015362613,-0.014124552,0.030565476,-0.017026674,-0.007987494,0.018211486,0.009338711,-0.004592809,-0.0038273407,-0.011382178,0.0054181833,0.022231858,0.08211143,-0.0025609902,0.008786243,0.0150031755,0.038579594,-0.019715797,-0.012666834,-0.012586959,0.0168403,0.006553073,-0.0034878722,-0.015535675,-0.024681356,-0.008500025,-0.016547425,-0.021566233,0.008333619,-9.5683517E-4,-0.021792546,-0.0060338858,0.023057232,-0.003427966,-0.0053749178,0.005900761,0.026704853,0.0080474,0.011701678,0.03929847,0.018690735,-4.975543E-4,-0.0060971202,0.0030602086,-0.007261963,-0.0061037764,-0.018251423,0.014909988,0.013725176,0.015389238,0.022178607,4.838258E-4,0.01807836,0.021193484,-0.0065963385,0.006962432,0.009365336,-0.01636105,-0.028382229,0.023616357,0.0025376934,-0.0090059,0.0172663,0.014444051,-0.0150031755,-0.026798042,0.0034878722,-0.001305456,-0.00925218,-0.008007462,-0.014909988,-0.025759667,-0.003236599,-0.033760473,-0.006070495,0.015855175,-0.03919197,-0.036369722,-0.031896725,-0.01152196,-0.020061921,0.01785205,-0.031151226,0.005594574,-0.010144117,-0.015841862,0.008320306,0.009817961,0.0163078,-0.011462053,0.011275679,0.0056078862,0.022657856,0.0029570367,0.015043113,-0.026664916,-0.0030019665,0.01098946,-0.0065996666,-0.00978468,0.0052384646,-0.019223236,-0.005954011,0.0024095606,0.006982401,-0.0022880842,0.0016690533,0.0036875596,0.003997075,0.0018737329,0.0038007158,0.007541525,-0.0138982395,0.006822651,-0.048404213,0.0011257374,0.009751399,-4.2121548E-5,-0.013079521,-0.0083669,0.020328172,-0.021499671,0.028408853,-0.022790981,0.0015558972,0.0060006045,-0.035783973,0.026292168,0.00820715,0.019502798,0.0035211535,-0.013445615,-0.032375976,-0.040736217,0.026758105,8.15806E-4,-0.015335988,0.0031151227,0.0035278096,-0.013265896,-0.02181917,-0.021632796,-0.0059706517,0.014776863,-0.013478896,-0.01604155,-0.027263978,-0.009385305,0.0085799,0.03458585,-0.017026674,-0.030059602,-0.013086177,0.009431899,0.0031583882,-0.008167212,0.022844233,-0.02587948,-0.025227169,0.0030801774,-0.015469112,0.032269474,0.0109628355,-0.01056346,-0.017705612,-0.0014718621,0.0034079973,-0.022924107,0.0056112143,0.024947606,-0.002068428,0.050161462,0.022631232,-1.3208485E-4,-0.01823811,0.0030485601,0.0010649991,0.0061337296,-0.026332105,9.3104225E-4,-0.017213048,3.1450758E-4,0.014617113,0.0031633803,0.013991427,0.010030961,-0.030671977,0.011455397,0.0050654025,-0.057190455,-0.004406434,-0.012766678,0.011714991,0.0058275424,0.004080278,0.011675053,-0.0041801217,-0.0020251626,0.014590489,0.0036043567,0.0069091823,0.005677777,0.020328172,-0.010716554,-0.0030851695,0.0024112246,-0.017758861,0.005597902,0.015841862,-0.018278047,-0.010603398,0.009611618,-0.0036276535,0.009758055,-0.002389592,-0.0122807715,-2.6832987E-4,0.013392365,-0.0151762385,-0.028568603,0.0064865104,-0.017931923,-0.026531791,-0.021606172,-0.033494223,-0.010070899,0.0042666527,-0.011675053,-0.0074017444,0.016374363,-0.00441309,-0.0162013,0.016613986,-0.007348494,0.025959354,-0.022471482,0.01058343,0.028941352,0.0066296197,-0.03192335,0.016187988,-0.0021682717,0.0034146535,1.6193396E-4,0.034399472,-0.0016698854,-0.01572205,0.024468357,-0.0027856384,-0.017332861,-0.041881092,0.022218546,0.014231051,0.023669606,-0.014563863,-0.014004739,-0.022524733,0.0072819316,0.016800363,-1.2563662E-4,-0.012427209,0.016667238,-0.030538851,0.028648477,-0.006063839,0.02171267,-0.0063999793,-0.0013420654,0.0012580303,3.3863646E-4,-0.029873226,-0.004433059,0.0018787251,0.016627299,-0.024641419,0.007255307,0.0024611466,-0.004616106,1.8627085E-4,0.022178607,0.021859108,0.031896725,-0.003887247,-0.012959708,0.0016141394,-0.012453834,0.0010974484,-0.040336844,-0.012167616,-0.01797186,-0.018464424,-0.009012556,-0.014364176,-0.0027939586,-0.006263526,-0.011908022,0.0058242143,0.0058541675,-0.024987543,-0.017665673,-5.330196E-5,-0.0036309815,-0.026425293,-2.0010336E-4,-0.015229488,-0.020021984,-0.015895113,-0.005328324,0.028488727,0.031736974,-0.024534918,-0.0074616503,-0.009425242,0.008699712,-0.030272601,-0.0042733094,8.653118E-4,0.004160153,0.009531743,0.0013287528,-0.028914727,-0.02373617,-0.0011448741,0.020714235,-0.0011332256,-0.0076280567,-0.009019212,0.00510534,0.025693106,0.020181734,-0.012360646,0.014510614,-4.750375E-5,-0.009338711,0.009072462,-0.021566233,0.012933084,0.013964802,0.012906458,0.005677777,-0.0026042557,-0.017306237,-0.024042357,0.005794261,-0.013725176,-0.018837173,-0.005008824,-0.0066495887,-0.003923856,0.011109273,0.008559931,-0.011036054,0.02540023,0.017093237,-0.012686803,-0.014137864,0.0056345114,-0.03362735,-0.030884976,-0.008320306,4.9921835E-4,-0.008506681,-0.011388835,0.016321111,0.010290555,0.0011215772,-0.0030602086,0.02123342,-0.005391558,0.003293177,0.004053653,0.010816398,-0.0057310266,0.006579698,0.015974987,0.027090916,-0.020927234,0.0019486157,0.026877917,-0.033254597,-0.008506681,-0.010610054,0.026292168,-0.022684483,-0.012826583,-0.027263978,-2.9474683E-4,-0.008453431,0.017891986,-0.008559931,-0.013146084,0.015974987,-0.029553726,0.023603044,0.011754928,0.010483585,0.005484746,-0.0014760223,-0.012653521,0.011295647,-0.013964802,0.021952296,9.144016E-4,-6.772729E-4,0.014377489,-0.0020301547,-0.029527102,-0.019449547,-0.01625455,-0.0090059,-8.457591E-4,0.02283092,-0.014031364,0.024588168,0.0027640057,-0.023922544,-0.01235399,0.0036376377,-0.018171549,-0.024455043,0.0049555744,0.0014177802,-7.33435E-4,0.009431899,0.024068981,0.0012205889,-0.0081539,-0.016480861,-0.008766274,1.251296E-5,0.011748272,-0.029287478,0.007721244,-0.014936613,-0.017013362,0.007155463,0.0031683724,-0.008034088,-0.013572083,-0.026398666,-0.028355602,0.009931117,-0.013399021,-0.018743984,0.013032927,0.010796429,-0.0028954665,0.015096364,0.20660985,0.023682918,-0.0038173564,0.036875594,0.029660227,0.0015184558,0.014470676,-0.003318138,0.017838735,0.023669606,0.022764357,0.028169228,-0.018837173,-1.8356675E-4,0.011954616,-0.022897482,-0.029260851,-0.0026342089,-0.0023446623,-0.034825474,-0.016640613,-0.0319766,-0.0023613027,0.010869647,0.034133222,-0.0031600522,6.385835E-4,-0.019382985,0.018650798,0.017199736,-0.012447177,-0.02540023,-0.011848115,-0.006862588,-0.0011939639,-0.009937773,-8.457591E-4,-0.014217739,0.018917048,-0.012200897,0.019689173,-0.004323231,0.008167212,-0.034692347,-0.0102373045,0.04523584,-0.007255307,-5.087867E-4,-0.0011265695,0.022231858,-0.024668043,-0.004349856,-0.0017189753,-1.7139831E-4,-0.010037617,-0.011229085,0.011881397,0.00282724,-0.020075234,0.006982401,-0.0062069483,0.010170742,-0.017026674,0.027397104,-0.028009478,0.014856738,-0.005005496,0.008074025,0.0159883,-0.012041147,0.005657808,-0.0037973877,-0.022577982,0.0033797084,-0.01657405,-0.020301547,0.006636276,0.035544347,0.0065430887,0.030352477,0.0016440925,-8.594876E-4,-0.00973143,0.010636679,-0.013186021,-0.021459734,0.004815793,0.012360646,0.006799354,0.003427966,-0.011848115,0.02203217,-3.1741968E-4,-0.016214613,0.0175858,-0.0073884316,0.016294487,0.015841862,-0.0081938375,0.014217739,-0.003933841,0.03732822,0.007947557,0.03477222,-0.02412223,-5.004664E-4,-0.023603044,0.031364225,-0.023616357,0.0053749178,0.020195046,0.0036908877,-0.008213806,0.007900963,0.013818365,0.020474609,-0.0024927636,-0.03940497,0.031124601,-0.0043365434,0.022751044,-0.037780844,0.011748272,5.033785E-4,-1.2990077E-4,-0.018783923,-0.010516867,0.02181917,-0.023216981,-0.044516966,-0.005095355,0.021100296,0.003923856,-0.009751399,-0.015948363,-0.0044530276,0.0021366547,-0.021060359,-0.012440521,-0.002229842,0.0017522564,-0.006716151,0.016773736,-0.013079521,0.028967977,-0.020248296,0.008293681,-0.014057989,-0.041801218,-0.01508305,-0.009212242,0.006476526,-0.011169178,0.0021599515,0.024841106,-0.012913115,-0.03818022,0.006117089,-0.010756492,-0.0107498355,-0.047898337,0.005468105,0.04914971,-0.013592052,-0.007648025,0.0038406532,-0.16805688,0.0360236,-0.006226917,-0.019276485,0.013592052,0.018104985,0.029873226,0.009125711,-0.0357041,0.020993795,-0.0067327917,-0.003293177,-0.025999293,-0.02116686,-0.0057243705,-0.017465986,-0.016653925,-0.005305027,-0.008227118,-0.0106633045,0.028488727,-0.012440521,0.041508343,-0.019156672,0.010809742,0.020461297,-0.0040969187,-0.0029770054,-0.00788765,-0.016720487,0.010649991,-0.027769852,0.014097926,-0.011129241,-0.0058574956,-0.022338357,-0.014031364,-0.023549793,-0.0097247735,0.024947606,0.020394733,0.0043631685,0.026718166,0.006193636,0.021047046,0.010483585,0.0019502797,-0.0032748724,0.007068932,-0.03967122,0.026678229,-0.011688366,0.018610861,0.016467549,0.011595178,-0.0017056627,-0.0064332606,0.017998487,-0.00407695,-0.00863315,-0.0044430434,-0.019795671,0.0024178808,-0.007261963,0.012626897,-0.023722857,-0.044942964,0.030991476,-0.028488727,0.010636679,0.0050121523,-0.01021068,-0.009778024,0.004299934,-0.008932681,0.029180977,-0.044090964,-0.01350552,0.0067128227,0.0059473547,-0.02165942,0.020461297,-0.039005596,-1.3988515E-4,-0.025746355,0.002015178,0.010523523,0.0031933335,0.010490241,0.0011573546,4.838258E-4,0.0076746503,0.014657051,-0.0035444503,-0.010257273,0.036343098,-0.022151982,0.022937419,-0.0044763247,-2.0163741E-5,0.01882386,-0.020660983,-0.039910845,0.012686803,0.028914727,7.213705E-4,0.0066129793,0.022910794,0.0029553727,-0.022524733,0.0021965608,0.026478542,0.00510534,0.020794109,-0.008892743,0.023123795,-8.644798E-4,-0.012121022,0.035331346,-0.0024245372,0.0304856,0.0021616155,-0.006692854,0.015322676,-0.014137864,-0.025227169,-0.11619141,-0.012633553,-0.011402147,0.011994553,-0.023909232,0.02582623,0.019875547,0.020195046,-0.0037574503,0.0020434672,0.013552114,-0.003374716,-0.0026624978,-0.009851242,0.026678229,-0.0013570419,-0.0046593715,-0.0051186522,-0.017825423,-0.0143242385,0.009631586,0.010596742,-4.8798593E-4,-0.017772174,6.951616E-4,-0.010277242,-0.04233372,0.01262024,0.0052417926,-0.0070090257,0.016321111,-0.0064598857,0.013924864,-0.020674296,-0.004516262,0.013139428,0.015522363,-0.008267056,0.025666479,-0.01983561,-0.0075881192,0.0151762385,0.006569714,-0.019808985,0.048350964,-0.007255307,-0.020354796,0.023123795,0.024415106,-0.009218899,-6.4232765E-4,-8.15806E-4,-0.028781602,-0.014164489,0.010843023,-0.014856738,0.014390801,0.009578336,-0.013565427,-0.0020651,-0.026824666,-0.007235338,-0.014696988,0.033494223,0.026092479,-1.1190811E-4,-0.03117785,-0.013166052,0.002038475,-0.029686851,-0.0061370577,-0.005471433,-0.008653118,0.0069757444,-0.020128485,0.01866411,-0.017252987,-0.025386918,-0.008599868,5.337476E-4,-0.012733396,-0.011115929,0.0016673893,-0.013086177,0.0062968074,0.014856738,-0.01882386,0.0087929,0.0059240577,-0.047232714,-0.0016682213,0.012973021,0.017146487,-0.020501234,-0.0075215567,0.0162013,-0.005408199,-0.0053349803,0.004795824,-0.015269426,-0.0326156,-0.013844989,-0.060225703,0.027370479,0.023017295,-0.023256918,0.008919368,-0.009877867,0.0019103423,-0.017758861,0.007315213,0.015495738,-0.00636337,0.020288235,-0.011435429,-0.0109628355,-0.009638242,-0.028541978,0.014816801,-0.0054547926,0.026917854,0.027237354,0.0025659823,-0.002444506,-0.0084734,0.02331017,-0.002258131,0.040150467,-0.040390093,0.029447228,0.0044463715,-0.017186424,-0.0077012754,-0.020195046,0.014510614,0.037887346,-0.014816801,0.005278402,-0.005960667,0.0077279,-0.012979677,-0.031204475,-0.021925671,-0.025080731,0.023869295,-0.0017356158,-0.011209116,-0.02685129,-0.041960966,0.013272552,0.034239724,0.034372848,0.046966463,0.010084211,-0.0090458365,-0.011954616,-0.007648025,8.191341E-4,0.025213854,0.026265541,-0.008806212,-0.0019985375,0.033334475,-0.013711864,0.012640209,-0.0025626542,0.015016488,0.0075149005,0.0041401843,-0.0011240733,0.018743984,-0.0023147091,-0.022524733,0.005764308,0.014670364,0.010916241,0.023057232,-0.011195804,-0.0025343653,-0.0059972764,0.0013462255,0.014617113,0.0027390448,0.0143242385,-0.012540366,0.048084714,0.0150031755,0.035491098,-0.023190357,0.022271795,0.004509606,-0.0025676463,-0.029766727,0.00473259,0.0016732136,0.00946518,0.017199736,0.0018271392,-0.020408047,0.0040902626,0.006759417,0.007182088,-0.005577933,0.01924986,-0.016161362,-0.009904493,-0.025946042,-0.016773736,-0.017252987,-0.029047852,-0.011195804,0.027556853,0.030805102,-0.039378345,0.0159883,7.1221817E-4,-0.0025942714,0.01373849,0.016294487,-0.008659774,-0.0136852395,0.033653975,0.014071302,0.009438555,0.016467549,-0.0062102764,-0.010483585,0.0068026823,-3.448767E-4,0.0021150217,0.011808178,-0.0062668542,0.012480459,-0.034053348,-0.019595984,0.01604155,-0.004602793,-0.015469112,0.04166809,0.03860622,-0.051785585,0.05410196,-0.0064099636,0.0074816193,-0.004376481,0.011541928,0.010729867,0.022577982,0.0062102764,-0.015735364,-0.017532548,-0.0138982395,-0.0143242385,0.034745596,-0.017878674,-0.012393928,0.027530229,-0.01058343,-0.009278805,0.006606323,0.0044996217,-0.003780747,0.0083669,-0.014577176,-0.0020517875,-0.024561543,-0.015202863,0.008300337,-0.028808227,0.008306993,-0.052184958,0.016866924,0.005983964,-0.057509955,-0.028568603,-0.00467934,-0.005963995,-0.013818365,0.004599465,-0.018903736,0.03075185,-0.0050154803,0.0074350256,-0.018038424,-0.018304674,0.015482426,0.0026508495,-0.014151176,-0.011115929,-0.0019719126
-0.010502041,-0.0057155085,0.0057155085,0.0053959675,-0.01076558,-1.3866698E-4,-0.009414942,-0.023560414,-0.026933717,-0.03555145,0.0069706147,0.032151796,-0.027513504,-0.012182104,0.003630254,0.008084068,0.02423244,0.008169718,-0.004967716,-0.030280666,-0.010798523,0.026894186,-0.0023982078,0.017525364,-0.004055211,0.0012839306,0.012195282,-0.018869415,0.007234154,0.013486624,0.008729739,0.013302146,-0.024627749,-0.0029812884,-0.011608906,0.00881539,0.008630912,-0.025589667,0.034497295,-0.028541306,-0.0031987084,0.014059822,0.014547369,-0.00890104,-0.019356962,0.0037982604,0.01471867,-8.2932523E-4,0.00911846,0.034918956,0.031018576,0.0035050728,-0.016378967,0.0106403995,0.0016561798,0.0018167741,0.0069376724,-0.0015458227,0.01076558,-0.020872314,4.0333866E-4,0.007148504,-0.0060054017,-4.970187E-4,-0.011213598,-0.009190933,-0.0052872575,-0.022466727,-0.031203054,0.001208163,-0.015351164,0.019475555,0.013256027,-0.012208458,0.020648304,-0.02522071,-0.015166687,0.02153116,0.0019864275,0.0032102382,0.021623401,-0.012827775,-0.0048524174,0.019725917,-0.0033897744,0.0033271837,0.0038839106,-0.0049215965,-0.0034358937,-0.014389246,0.012900249,0.009138226,0.005992225,-0.010086967,-0.022242717,0.030148897,0.0056726835,0.021465277,-0.009138226,-0.017380416,-0.002744103,0.010785346,-0.0128738955,-0.0096060075,-0.0154961115,0.013769928,-0.015957305,-0.020160757,0.012742125,-0.015048094,0.010284621,0.009045986,-0.0034194225,-0.013756752,0.0076228743,0.018342335,0.0011208656,0.004424166,0.015061271,-0.034550004,-0.0026551585,0.020674659,0.021992356,-0.0072802734,0.009480827,0.009876136,-0.022914743,-0.0030224663,0.010877585,0.012663064,0.022914743,0.04095401,0.0125708245,-0.019633679,-0.0030652916,0.0040090918,-0.0031772957,0.016339436,-0.007148504,-0.01867176,0.019567793,0.04975622,-0.022071417,0.012827775,-0.0060054017,0.015654234,0.004928185,0.07073395,-0.010877585,-0.019185662,0.015337988,-0.0039366186,0.010001317,-0.01498221,0.001153808,0.017090524,0.0055540907,0.023942545,9.610949E-4,-0.013506389,0.019897217,0.015337988,-0.007853472,-0.020964552,0.010416391,0.034629066,0.023955723,-0.034708127,-0.007332981,-0.019976279,-0.026867833,0.0011035708,-0.016747924,0.0054420866,0.015614704,-0.0047041764,0.025919091,-0.0060449326,-0.021214914,-0.010614046,0.006021873,-0.021623401,0.01346027,0.00994202,-0.0051060743,-0.011694557,0.015258926,7.5438124E-4,0.033627614,-0.022809327,0.026828302,0.027566211,-0.03976808,-0.032863352,-0.6544208,-0.028198706,0.00794571,-0.018078797,0.004447226,0.035709575,-0.011977862,0.036315717,0.0037455524,0.022110948,-0.011931743,0.0027111606,0.012089865,9.479179E-4,0.0017541836,-0.00963895,0.017314533,-0.0036961387,0.017828435,0.0020407825,-0.003742258,0.023823954,0.010488864,-0.016378967,0.018315982,0.0018118328,0.011009354,-0.0146791395,-0.01971274,0.027513504,-0.009454473,0.0053959675,0.012452233,-0.002607392,0.058927387,0.008321254,0.014072999,0.004915008,0.023639476,0.016747924,-0.018250097,-0.028857553,0.006074581,-0.012063512,-0.019053891,0.0064501246,0.026393462,0.014824086,-0.001788773,-0.015693765,0.0058571612,-0.0064072995,-0.013704044,0.008505731,0.0023487941,-0.0017953615,0.028409537,0.0092634065,5.1215156E-5,0.016418498,-0.0021461982,-0.0080181835,-0.01354592,-0.022282248,0.007840294,0.034550004,0.004980893,0.013585451,0.02205824,-0.011925153,0.01198445,0.024087492,-0.021952825,0.021175383,0.003778495,0.027618919,3.174825E-4,0.01802609,0.0034556591,6.8643753E-4,6.250823E-4,-0.0029466988,2.1330212E-4,0.0034984844,0.024733163,0.004911714,-0.013888521,0.008782447,8.404433E-4,-0.013005665,-0.0073922775,0.007524047,-0.0032794173,-0.0042462773,0.0024476212,0.006483067,-0.008749505,0.006726841,0.010818289,-0.007668994,0.013743575,-0.0013679337,-0.0075372243,0.0025596255,0.013440505,0.024351032,-0.012913426,-0.004453814,0.015878243,0.0039069704,0.038450386,0.006433653,-0.015627882,-0.024087492,-0.0191066,-0.028198706,0.02775069,0.01241929,0.0058999863,-0.012992488,0.0022713793,0.015627882,0.007939122,0.020358412,-0.0026996308,0.036157593,-3.069821E-4,0.009718012,0.0040881536,-0.026327576,-0.0010335682,-0.017314533,-0.013532744,-0.003834497,-0.0065258923,0.030148897,0.025207534,-0.0043615755,0.022545788,-0.012696006,-0.0096060075,0.008373962,0.006595071,7.8773545E-4,0.009461061,-0.021004083,-0.01649756,-0.0074713393,-0.011839503,0.029542755,-0.012425878,-0.010416391,-0.020977728,0.022216363,-0.026498877,-6.835551E-4,-0.0076821707,-0.014072999,-5.752569E-4,-0.008953748,0.019660031,0.03273158,-0.046145733,-0.008176307,0.0110686505,0.00868362,0.015298457,0.005007247,-0.0064105936,-0.017788904,0.019660031,-0.024772694,-0.007840294,0.013005665,-0.0101660285,0.009876136,-0.021781523,-0.0030027009,0.013928052,0.011997627,0.018012911,0.028620368,-0.0123599935,0.014692317,0.023547238,0.011417841,0.0064501246,0.017564895,-0.02527342,0.008037949,-0.014481485,0.010620634,0.002376795,0.009823428,-0.02088549,-0.030122543,0.010264856,-7.3554234E-5,0.0027951638,0.008308076,0.014007114,0.010442745,0.010706284,-1.0654812E-4,0.019343786,-0.028804846,0.0241402,-0.014007114,0.005435498,0.011167478,0.028752139,2.0043399E-4,0.0031690602,-0.0070496765,-0.016207667,0.013256027,-0.008327842,0.024495978,3.967502E-4,-0.0288312,-0.018645406,-0.015733296,0.0011076886,-0.008005006,-0.018777175,0.01793385,0.025905915,0.0062162336,-0.019910395,-0.016075898,0.023072867,0.005162076,0.003999209,0.023494529,0.017182764,-0.026037684,0.039241,-0.023823954,0.010620634,0.006812491,-0.005122545,0.027039133,0.0077282903,0.0024393857,0.026248515,0.011951507,0.022611672,0.015034918,-0.00851232,-0.010567926,0.023389114,0.017920673,-0.020252995,-0.0073198043,0.010324152,-0.015996836,0.0053267884,0.003630254,0.030913161,0.0054816175,0.00942153,-0.0065621287,0.025326127,0.010600869,0.033759385,-0.005936223,-0.026788771,-0.012735537,0.021676108,0.022242717,0.008308076,-0.020964552,0.0016850044,-0.016115429,-0.008545262,0.006878376,-0.001569706,-0.013117669,0.016602976,-0.022941096,-0.034971666,-0.01059428,0.047621552,0.0062985895,-0.014376069,-0.007398866,-0.0055936216,-8.18619E-4,-0.013809459,0.010528395,0.008841744,0.02296745,-0.004371458,0.01897483,-0.012906837,-0.0036039,0.017854787,0.008670443,-0.0016611211,-0.011312425,0.012399524,-0.006427065,-0.010100144,0.0020654893,0.012076689,0.03573593,-0.04419554,-0.0070628533,-0.003124588,0.0023537355,0.019291077,-0.0034886017,-0.0029812884,0.011997627,0.019541439,0.0060844636,-0.015733296,7.103208E-4,0.02171564,0.015140333,0.003778495,-0.042324413,-0.02584003,0.01324285,0.0804322,0.024825403,-0.009764131,-0.0014527604,-0.0032267093,0.00859797,-0.03934642,0.030017126,-0.013730397,-0.002169258,-0.027170902,-0.008828566,0.008966925,0.0078007635,0.010884173,0.004269337,-0.0064797727,-0.024258792,0.0018134798,-0.030755037,-0.0038872047,-0.018012911,-0.018105151,0.01940967,-0.005198313,0.010317564,0.027065488,0.010706284,0.023784423,-3.9942676E-4,-0.0028972852,0.008716563,0.01632626,-0.008064303,-0.016313083,8.997397E-5,-0.0017854788,0.003508367,0.0019732506,0.00169324,0.014231122,0.015074449,0.0129595455,-0.0194492,0.012412702,-0.02809329,0.008525496,0.03520885,0.024627749,-0.0019501909,0.04011068,-0.006631308,0.007629463,0.016062722,0.016313083,-0.0145869,5.583739E-4,-0.009461061,-0.010001317,-0.018698113,0.005896692,-0.034971666,0.007794175,-0.021201737,-0.027618919,-0.018724468,-0.02956911,5.839043E-4,-0.013585451,0.004654763,-0.0021116086,-0.0064797727,-0.009026221,-0.003482013,0.03130847,-0.015904598,0.01806562,-0.0031179995,0.015403872,0.027882459,0.009592831,-0.015048094,-0.023507707,-0.034813542,0.0013498154,0.020701012,0.015219395,-0.009869547,-0.012794834,-0.009355646,-0.008373962,-1.1848974E-4,0.0064896555,-0.013769928,0.009381999,-0.014797732,0.010752404,0.0014247594,0.03225721,-0.0059329285,-0.0068586105,0.0066675446,-0.005610093,-0.014665962,0.009441296,-0.017472656,-0.040005267,-0.0065258923,0.0017014757,0.004368164,0.017275002,-0.018474106,-0.019356962,7.2308595E-4,-0.006483067,0.0036697849,0.003030702,0.020173933,0.0035610748,-0.001869482,-0.009368822,-0.013104492,-8.2273677E-4,-0.010040848,-0.008420081,0.019699562,-0.006697193,-0.07758597,-0.012742125,-0.010515219,0.0016001777,-0.012063512,-0.011734088,0.019172484,-0.009586242,-0.019356962,-0.007326393,0.014652786,-0.00338648,-0.03565687,-0.0092634065,-0.009052576,-0.021689285,-0.0010829818,-0.012267754,-0.032125443,-0.02357359,-0.013572275,-0.022703912,0.038397677,0.012386347,0.013236262,-0.014389246,0.0031641189,0.0029928181,-0.022176832,0.007675582,0.013084726,0.028304122,0.030860452,0.025945446,-0.0022911448,-0.0063710627,-0.009408353,-0.002185729,0.0034227166,0.013032019,-0.028514953,-0.029410986,7.71676E-4,0.016668862,0.0030554088,0.030069835,0.015048094,0.0015384107,-0.010198971,0.014072999,-0.01181315,-2.4150907E-4,-0.026288046,0.010607457,-0.012247989,-0.015641058,0.033680324,0.003962972,-0.0148504395,0.004068388,0.00890104,-0.01176703,-0.018935299,0.039082877,-0.021807877,0.009744366,0.013532744,-0.0013926405,-0.003098234,-0.01958097,-0.01702464,-0.00903281,0.007517459,0.015614704,0.009777308,-0.008742916,-0.017275002,-0.0034556591,0.008110422,-0.017261824,0.013624982,-0.015667412,-0.022888388,-0.04237712,-0.03299512,-0.028804846,-0.042930555,0.019646855,-0.011266305,0.0029532874,0.017828435,-0.030623266,-0.031624716,-0.026327576,-0.03439188,0.03433917,-0.0065621287,0.027170902,0.026419815,-0.0143365385,-0.025471075,0.005827513,0.011582552,0.013730397,0.024825403,0.0033189482,0.0095533,-0.011305836,-0.0117143225,-0.021109499,-0.008130187,-0.026696531,-0.0012411054,-0.027513504,0.006891553,-0.011272894,-0.018908946,-0.021636577,0.0060119904,-0.011720911,-0.0065983655,-0.0011653379,-0.017406771,-0.031229408,0.015272103,-0.003738964,0.036447488,0.019831333,-0.016892869,-0.014297008,0.00412439,-0.04867571,0.011193832,0.03078139,-0.0014025233,-0.0062722354,0.010864408,0.0068322564,0.011272894,0.0031690602,-0.010370271,-0.01702464,0.023758069,-0.031940963,-0.0017690076,0.010152852,0.01653709,0.008189484,0.020226642,-0.009592831,-0.031677425,-0.0011571023,-0.018829884,0.0165898,0.022769796,-0.017011462,-0.0073527466,-0.0139939375,-0.009513769,-0.04137567,-0.022717088,0.010838054,-0.0017937144,-0.0022219657,0.0013951112,-0.026643824,-0.0028198706,0.005573856,0.0027671626,-0.002335617,-0.00477665,-0.022638027,0.02062195,-0.0064929496,-6.164349E-4,-0.045038868,0.011786795,0.0022005532,-0.02809329,0.018250097,-0.019607324,-0.037659768,-0.0039135586,0.0038937933,0.017696664,-0.0074186316,0.008802213,0.03465542,-0.0054256152,6.267294E-4,8.614441E-4,-0.006499538,0.016102253,-0.01324285,0.003478719,0.0048128865,-0.019752271,-0.020687835,-0.0015046448,0.0151535105,-8.8368024E-4,-0.02279615,-7.3008623E-4,0.005531031,0.03555145,0.001017097,-0.021293975,-0.038450386,-0.007939122,-0.0073659234,0.009948608,0.0059296344,-0.005652918,-0.0010870997,0.009586242,-0.0048820656,0.012083277,-0.0029466988,-0.015430226,-0.02470681,-0.012392936,-0.014362892,-0.0074054543,-0.0010162734,0.03204638,0.010146263,-0.028014228,-0.031545654,-0.020569243,-0.026261691,-0.0022285543,-0.033627614,-0.006404005,-0.0034853073,0.012768479,-0.0042462773,0.024047961,-0.0025629199,-0.004410989,-0.0041375672,-0.022769796,0.0015285279,0.0064468305,-0.012195282,-0.0047173537,-0.0054947943,-0.008426669,0.015258926,0.020951375,-0.017314533,-0.0034095398,-0.0015507641,-0.010752404,-0.0136908665,0.026261691,0.028857553,-0.014903148,0.026380284,-0.028910263,-0.024087492,-0.017551718,0.012379759,-0.016853338,-0.008327842,0.0050303065,0.0051060743,0.0022565552,-0.006348003,0.008031361,-0.00794571,-0.00942153,0.009658716,0.009085517,-0.033416785,0.0056726835,-0.013572275,-0.021333506,-0.0033469491,1.3475506E-4,-0.015614704,-0.021399392,0.025747791,0.0056562126,-0.032072734,0.0014708787,0.024456447,0.0057286858,-0.0025777437,-0.010956647,-0.008973514,-0.0145869,0.029806295,-0.0057385685,-0.0062689413,-0.008202661,-5.8472785E-4,7.745585E-4,0.022875212,-0.028488599,0.01146396,-0.024983527,0.001543352,0.002785281,-0.02500988,-0.045276053,-0.014231122,-0.009790485,-0.043273155,0.004522993,0.20071153,3.983973E-4,0.014995387,0.041217547,-0.018171035,0.014758201,0.019119777,0.011437606,0.0020654893,-0.01228752,0.024192909,0.032942414,-0.00564633,0.0039432067,0.0145078385,-0.016444853,-0.0108117005,-0.039504543,0.0069310837,-0.034760833,-0.0020094872,3.300418E-4,-0.014639609,-0.011780207,0.033891156,-0.013822637,-0.016682038,0.013144023,0.01337462,0.0074318084,-0.018302806,-0.016826985,0.030254312,-0.010910527,-0.010284621,-0.0039366186,0.0026996308,0.020305704,0.009909078,0.041955456,0.03460271,-0.009974963,0.015561996,-0.0169324,-0.026077215,0.026604293,-0.0018876003,-0.010238502,0.0071616806,-0.012643298,-0.010719461,-0.0013695809,0.008420081,0.036104884,-0.005570562,0.004859006,0.021952825,0.006321649,-0.018605875,0.015561996,0.0038542624,0.01367769,0.0056660953,0.0311767,-0.01211622,0.023955723,-0.034760833,-0.0062195277,-0.020200288,-0.005422321,0.01046251,0.002238437,-0.0067466064,-0.006624719,-0.00786006,-0.009316115,0.017459478,0.014007114,0.023599945,0.014771378,6.226116E-4,-0.026867833,-0.046356563,-0.0048919483,-0.0041441554,-0.017130055,0.035340622,-0.012478586,0.015772827,-0.021900116,0.0144156,-0.023336405,-0.009764131,-0.0027506915,0.013783106,0.032336272,-0.007108973,0.016642507,-0.026973248,-0.026367107,-0.0093490565,0.017551718,0.014468308,0.005764922,-0.01802609,-0.014125707,0.0037488467,0.025154827,0.025642375,0.0026139806,0.004476874,-0.0066115423,0.00920411,-0.023323229,0.005992225,0.012208458,-0.037501644,-0.020226642,0.022084594,-0.02275662,0.0100803785,-0.0060679927,0.006496244,0.01753854,-0.0135195665,-0.016207667,-0.021676108,0.023375936,-0.011958096,-0.014257477,0.019436024,0.008123599,0.002811635,-0.0063117663,-0.01411253,-0.021821054,-0.014323361,0.0105547495,0.0025480958,4.879595E-4,2.7959872E-4,0.014402423,0.024351032,0.005092897,0.0028643429,-0.010034259,0.01176703,-0.0035248382,0.0017245354,-0.004071682,-0.010838054,-0.042851493,-0.021702461,0.005339965,0.0052477266,0.013954407,-0.010258268,-0.033100538,-0.013249438,-0.0019040715,-0.0288312,0.002307616,0.029938065,0.0016314731,-0.015245749,0.0046481746,-0.16729476,0.019001184,0.01988404,0.0050303065,0.005814336,-0.012821187,0.02249308,0.010449333,-0.032942414,0.023652652,0.013051785,-0.011114771,0.0059724594,-0.0020869018,0.004872183,-0.023375936,-0.017406771,0.010567926,0.021597046,-0.0039069704,0.025774144,0.004750296,0.022558965,-0.011437606,0.0019221898,0.01819739,2.8392242E-4,0.007102384,-0.019515086,-0.009797074,0.009718012,-0.01706417,0.015838712,-0.0076624053,0.03887205,0.0014338185,-0.02171564,0.004898537,-0.008492555,0.016260374,0.003877322,0.0059625767,0.0083607845,0.0040980363,0.015812358,0.0044669914,-0.014231122,0.0018497165,-0.011299248,-0.030939514,0.010060613,-0.031545654,0.0011381604,0.0076558166,0.018184213,0.022782972,0.009302937,0.026815124,-0.0031031754,-0.0031476475,7.1114436E-4,-0.01228752,-0.012425878,-0.017393595,-0.009573066,-0.03726446,-0.015575173,-0.0351825,-0.03404928,0.0047700615,-0.007840294,0.004186981,0.01346027,0.014191591,0.027197257,0.004793121,-0.03273158,-0.0037686122,-0.01736724,0.010482276,-0.037448935,0.027381733,-0.029806295,0.012920015,-0.010844642,0.01233364,-0.017433126,0.009599419,0.027460795,0.017169585,-0.0039135586,0.0017360651,-0.013018842,-0.029463693,0.02835683,0.02522071,-0.008775859,0.0034984844,0.0075701666,-0.006838845,-0.0048952424,0.005580445,-0.02709184,0.054605342,0.0072802734,0.010844642,9.932137E-4,3.6586667E-4,0.017617602,-0.035235204,-0.0026156276,0.027724335,-0.021083144,0.04311503,0.015654234,0.014995387,-0.009711424,-0.022835681,0.048306756,-0.016721569,0.05310317,-0.015996836,-0.01302543,0.015917774,0.014652786,-0.01124654,-0.10019764,-0.01698511,0.009316115,0.02279615,-0.03360126,-0.0047305305,0.013064961,0.015825536,-0.0072275656,0.016471207,-0.01572012,-0.0059164576,0.017169585,-0.0053992616,0.039399125,-0.014217946,0.017657133,0.004345104,-0.017683487,0.019080246,-0.01719594,-0.013756752,-0.01958097,-0.035103437,0.0065225977,-0.001125807,-0.029911712,0.014033468,0.022954274,-0.020793252,0.011015943,-0.021254444,-0.008123599,-0.0145078385,0.027039133,0.020042164,-0.035630513,0.02417973,0.02983265,-0.04008433,0.007978653,0.010495453,-0.020463828,-0.014652786,0.0041013304,0.0074647507,-0.019053891,0.014626431,8.037949E-4,0.0051521934,-0.0082751345,0.024640925,-0.0010343918,-5.031954E-4,0.03760706,0.02253261,9.462708E-4,0.03104493,-0.013321912,-0.019488731,0.004437343,0.006374357,-0.0082751345,0.005804453,0.016444853,0.013150611,-0.0035841346,8.4662E-4,0.013269204,-0.020160757,0.01293978,1.6121194E-4,0.0015812358,0.022677558,-0.019396493,0.0028610486,-0.025853205,-0.015996836,0.008472789,-0.02088549,-0.011589141,-0.004486757,-0.008558439,-0.007978653,-0.013888521,0.007082619,0.018961653,-0.015706943,0.02357359,-0.037000917,-0.018447751,-0.0042462773,0.027118195,0.015087625,-0.011944919,0.01428383,0.008802213,-0.006950849,0.009764131,-0.010627222,-0.02674924,-0.028725784,-0.05845302,0.029279217,0.012023981,-0.0068322564,-0.020569243,0.008848332,0.0108973505,-0.0062854122,-0.0064138877,0.016853338,-0.025207534,0.032889705,-0.018184213,-0.016286729,-0.0108512305,-0.020252995,-0.0025283303,-0.006621425,0.025233889,-0.007629463,0.01572012,-0.018276451,0.0125708245,0.002674924,0.0028709313,0.0078007635,-0.02153116,0.036368426,-0.002564567,-0.012715772,0.02692054,0.003920147,-0.0013201672,0.006499538,-0.008578205,0.0025760967,-0.011009354,0.020635128,0.01984451,-0.013256027,-0.01302543,-0.028066937,0.0105547495,-0.020582419,-0.049018312,0.006196468,-0.010040848,0.0076624053,0.025892736,-7.6344045E-4,0.03773883,0.032626167,0.004628409,-0.025365658,-0.041112132,-0.00920411,0.036500193,0.0067367237,-0.006021873,-0.025984976,0.03057056,-0.004602055,0.014692317,-0.02357359,0.011648437,0.009803662,0.0056595067,0.011543022,-0.0034622476,-0.007412043,0.013888521,-0.0135195665,0.008657266,-0.0331796,0.025734613,-0.007029911,0.013473447,-0.018763999,-0.031598363,0.0076624053,-0.0065160096,0.0044834623,-0.035973117,-2.5921562E-4,-0.010442745,0.0165898,-0.008657266,0.013124257,-0.01923837,-0.0077875867,-0.03391751,0.019976279,-0.014257477,-0.0063809454,0.010403214,0.02192647,-0.0064797727,0.0092634065,0.025378834,0.010034259,0.009428118,0.00786006,0.01168138,0.008518908,-0.029648172,-0.015206218,-0.022730265,-0.03520885,0.027249964,0.014165238,0.0024410328,-0.04430096,-0.006852022,0.011345367,-0.016892869,0.01836869,-0.0069837915,0.004345104,-0.027170902,0.031677425,-0.003916853,0.008611147,0.026143098,-0.01884306,0.024061138,-0.003127882,-0.019936748,-0.017261824,0.022690734,-0.009230464,0.0062755295,0.009322703,0.0082751345,0.002062195,-0.0058341012,0.0025480958,0.021346685,0.010752404,-0.004414283,0.032889705,-0.015074449,0.00516537,-0.0020078402,0.008222426,0.03225721,0.018487282,0.0058242185,0.018263275,4.1332434E-5,0.0026008035,-0.005050072,0.028435891,-0.035630513,-0.020345235,0.014099353,-0.0012007509,-0.0030883513,-0.023705361,0.022018708,0.028304122,0.009414942,0.012175516,0.005214784,-0.01958097,-0.0029499931,-0.0037455524,-0.023296874,-0.0017311238,-0.03217815,0.005310317,0.003837791,-0.03676373,-0.01702464,0.022888388,0.00786006,-0.0070233224,0.015008563,-0.004381341,-0.024719987,0.006812491,0.0385558,-0.02010805,0.005610093,0.0019205427,-0.027408088,-0.018355513,-0.028989324,-0.022466727
I've put that whole file on GitHub as well.
As you can see, though, each line consists of a series of floating point numbers separated by commas. Also, each line corresponds to a single line in the input file which you saw above.
Best of all: the lines are in the same order. So the first line in the Embeddings file corresponds to the first line in the input file. The second line in the Embeddings file corresponds to the second line in the input file. And so on.
SearchDataSetHelper
The SearchDataSetHelper class makes it easy to instantiate a SearchDataSet object.
public class SearchDataSetHelper {
private static Logger LOG = LoggerFactory.getLogger(EmbeddingsHelper.class);
public static final SearchDataSet createSearchDataSet(final String inputFile, final String embeddingsFile) throws IOException {
preCreateCheck(inputFile, embeddingsFile);
final SearchDataSet dataSet = new SearchDataSet();
dataSet.setInputFile(inputFile);
dataSet.setEmbeddingsFile(embeddingsFile);
return dataSet;
}
private static void preCreateCheck(final String inputFile, final String embeddingsFile) throws IOException {
final Path inputFilePath = Paths.get(inputFile);
if (Files.notExists(inputFilePath)) {
throw new IOException("File " + inputFile + " doesn't exist!");
}
LOG.debug("Input file path " + inputFile + " exists");
final Path embeddingsFilePath = Paths.get(embeddingsFile);
if (Files.notExists(embeddingsFilePath)) {
//if we get here, the embeddings haven't been saved yet
//we'll save them now
LOG.debug("No embeddings file path, creating one...");
final List<String> list = new ArrayList<>();
try (Stream<String> stream = Files.lines(inputFilePath)) {
stream.forEach(line -> list.add(line));
}
final boolean saved = EmbeddingsHelper.persistEmbeddings(list, embeddingsFile);
if (!saved) throw new IOException("Problem saving embeddings!");
else LOG.debug("Saved embeddings file path successfully");
} else {
LOG.debug("Embeddings file path " + embeddingsFile + " already exists");
}
}
}The public static method at the top accepts two parameters:
- The input file location - the plain text that you want users to search in
- The Embeddings file location - the file that includes the Embeddings corresponding to the input file
And remember: you've seen samples of both of those files already.
The preCreateCheck() method accepts the same two parameters. It starts off by validating that the first file exists.
Then, it checks if the second file exists. If that file does not exist, it will go out to OpenAI and get the Embeddings for the input file. Then, it will persist those Embeddings at the specified location.
SearchTermHelper
The SearchTermHelper class makes it easy to get the Embeddings for a given keyword or search query.
public class SearchTermHelper {
public static List<Double> getEmbeddingForSingleSearchTerm(final String searchTerm) {
final List<String> searchTermAsList = Arrays.asList(searchTerm);
final List<Embedding> embeddings = EmbeddingsHelper.getEmbeddings(searchTermAsList);
//should only get 1 result for a single search term
return embeddings
.get(0)
.getEmbedding()
.stream()
.collect(Collectors.toList());
}
}That method transforms a single search expression into a List that includes just one item. Then, it goes out to OpenAI and gets the Embedding for that list.
Finally, the method converts the List of Embedding objects to a List of Double objects. Then it returns that list.
SemanticSearchHelper
The last support class that for today's tutorial is the SemanticSearchHelper class. It streamlines the search process.
public class SemanticSearchHelper {
private static Logger LOG = LoggerFactory.getLogger(SemanticSearchHelper.class);
private static final int DEFAULT_MAX_RESULTS = 5;
private final SearchDataSet dataSet;
private final List<String> inputs;
private final List<List<Double>> dataSetEmbeddings;
private SemanticSearchHelper(final String inputFile, final String embeddingsFile) throws IOException {
this.dataSet = SearchDataSetHelper.createSearchDataSet(inputFile, embeddingsFile);
this.inputs = dataSet.getInputAsList();
this.dataSetEmbeddings = EmbeddingsHelper.loadFromFile(dataSet.getEmbeddingsFile());
}
public static SemanticSearchHelper getSemanticSearchHelper(final String inputFile, final String embeddingsFile) throws IOException {
return new SemanticSearchHelper(inputFile, embeddingsFile);
}
public List<SearchResult> search(final String searchTerm, final int maxResults) {
//first, get the embedding for a single search term - this returns a list of 1
final List<Double> searchEmbedding = SearchTermHelper.getEmbeddingForSingleSearchTerm(searchTerm);
//instantiate an array where we'll store the results
final List<SearchResult> results = new ArrayList<>();
//step through all the content we want to search for
for (int i=0; i<dataSetEmbeddings.size(); i++) {
//get the cosine similarity between the search term and this part of the content
final double score = Calculations.cosineSimilarity(dataSetEmbeddings.get(i),
searchEmbedding);
//instantiate a SearchResult object that includes
//the content and the score
final SearchResult searchResult = new SearchResult();
searchResult.setScore(score);
searchResult.setText(inputs.get(i));
results.add(searchResult);
}
//sort the results - this will be in DESCENDING order
Collections.sort(results);
//return the maximum number of results we're looking for
return results.subList(0, maxResults);
}
public List<SearchResult> search(final String searchTerm) {
return search(searchTerm, DEFAULT_MAX_RESULTS);
}
}Unlike the other support classes, this one gets instantiated.
The getSemanticSearchHelper() method accepts two parameters that you're quite familiar with:
- The input file location - the plain text that you want users to search in
- The Embeddings file location - the file that includes the Embeddings corresponding to the input file
It uses those two parameters to instantiate and return a SemanticSearchHelper object.
The overloaded search() method accepts the search term as a string. It can also accept the maximum number of results to be returned. The default number of search results, based on the DEFAULT_MAX_RESULTS constant above, is 5.
Next, the search() method gets the Embeddings for the search term. You've already seen that code.
Then the code steps through the entire data set of available Embeddings and checks the cosine similarity against the search term.
Within that loop, the code constructs a new SearchResult object and stuffs it with the plain, readable text from the input file and the score that got calculated by cosine similarity.
That SearchResult object gets added to the results list.
Once the loop is finished, results gets sorted. Remember, though, it's sorted in DESCENDING order because you want the results with the highest scores at the top.
Finally, the method returns a sublist that includes no more results than the number specified in maxResults.
Running It
Now that you've got all the support files in place, the rest is a snap. Just create a simple class that runs a semantic search.
public class SemanticSearch {
private static Logger LOG = LoggerFactory.getLogger(SemanticSearch.class);
private static final String INPUT_FILE = "./amazon-food-reviews.txt";
private static final String EMBEDDINGS_FILE = "./amazon-food-reviews-embeddings.txt";
private static final String SEARCH_TERM = "good taffy";
public static void main(String[] args) {
try {
final SemanticSearchHelper semanticSearchHelper = SemanticSearchHelper.getSemanticSearchHelper(INPUT_FILE, EMBEDDINGS_FILE);
final List<SearchResult> results = semanticSearchHelper.search(SEARCH_TERM);
results.forEach(result -> System.out.println(result.getText() + " " + result.getScore()));
} catch (Exception e) {
LOG.error("Problem with semantic search!", e);
}
}
}That class starts off by defining a couple of files:
- The input file location - the plain text that you want users to search in
- The Embeddings file location - the file that includes the Embeddings corresponding to the input file
Hopefully by now, that's all familiar territory to you. And remember, I've put both of those files in GitHub so you can check them out.
The next constant defines the search term. I'm just keeping things simple here by hardcoding the query.
Within the main() method, the code starts by instantiating a SemanticSearchHelper class.
In the next line, the code invokes the search() method on that SemanticSearchHelper object with the hardcoded query. The response gets returned as a List of SearchResult objects.
Finally, the code prints out the search results in order of relevance. The code also prints the score associated with each search result.
Now run that code above and check the console. You'll probably see results that look like this:
Title: Wonderful, tasty taffy; Content: This taffy is so good. It is very soft and chewy. The flavors are amazing. I would definitely recommend you buying it. Very satisfying!! 0.896129307546035
Title: Great taffy; Content: Great taffy at a great price. There was a wide assortment of yummy taffy. Delivery was very quick. If your a taffy lover, this is a deal. 0.8872214596115646
Title: Nice Taffy; Content: I got a wild hair for taffy and ordered this five pound bag. The taffy was all very enjoyable with many flavors: watermelon, root beer, melon, peppermint, grape, etc. My only complaint is there was a bit too much red/black licorice-flavored pieces (just not my particular favorites). Between me, my kids, and my husband, this lasted only two weeks! I would recommend this brand of taffy -- it was a delightful treat. 0.8781955974435804
Title: Great! Just as good as the expensive brands!; Content: This saltwater taffy had great flavors and was very soft and chewy. Each candy was individually wrapped well. None of the candies were stuck together, which did happen in the expensive version, Fralinger's. Would highly recommend this candy! I served it at a beach-themed party and everyone loved it! 0.8749170225608228
Title: fresh and greasy!; Content: good flavor! these came securely packed... they were fresh and delicious! i love these Twizzlers! 0.8496450881850831
And that seems right. You got 5 results and the top one sure looks like a good match for "good taffy."
Now change the SEARCH_TERM constant to "great spicy sauce" and run it again. Here's what I got:
Title: The Best Hot Sauce in the World; Content: I don't know if it's the cactus or the tequila or just the unique combination of ingredients, but the flavour of this hot sauce makes it one of a kind! We picked up a bottle once on a trip we were on and brought it back home with us and were totally blown away! When we realized that we simply couldn't find it anywhere in our city we were bummed.<br /><br />Now, because of the magic of the internet, we have a case of the sauce and are ecstatic because of it.<br /><br />If you love hot sauce..I mean really love hot sauce, but don't want a sauce that tastelessly burns your throat, grab a bottle of Tequila Picante Gourmet de Inclan. Just realize that once you taste it, you will never want to use any other sauce.<br /><br />Thank you for the personal, incredible service! 0.8443179719417426
Title: fresh and greasy!; Content: good flavor! these came securely packed... they were fresh and delicious! i love these Twizzlers! 0.8001285844983123
Title: Delicious product!; Content: I can remember buying this candy as a kid and the quality hasn't dropped in all these years. Still a superb product you won't be disappointed with. 0.7781411436264448
Title: Great taffy; Content: Great taffy at a great price. There was a wide assortment of yummy taffy. Delivery was very quick. If your a taffy lover, this is a deal. 0.7705765535164092
Title: Wonderful, tasty taffy; Content: This taffy is so good. It is very soft and chewy. The flavors are amazing. I would definitely recommend you buying it. Very satisfying!! 0.7699669590639245
That top result looks quite relevant. And take note that the phrase "spicy sauce" doesn't occur in the result. But the phrase "hot sauce" does.
Also the search results got the sentiment right: the reviewer said he really liked that hot sauce. That matches the "great" part of the search term.
Try one more experiment. Change the query to "cats like the weight loss food." Run it again. Here's what I got:
Title: My cats LOVE this "diet" food better than their regular food; Content: One of my boys needed to lose some weight and the other didn't. I put this food on the floor for the chubby guy, and the protein-rich, no by-product food up higher where only my skinny boy can jump. The higher food sits going stale. They both really go for this food. And my chubby boy has been losing about an ounce a week. 0.8820211269561004
Title: My Cats Are Not Fans of the New Food; Content: My cats have been happily eating Felidae Platinum for more than two years. I just got a new bag and the shape of the food is different. They tried the new food when I first put it in their bowls and now the bowls sit full and the kitties will not touch the food. I've noticed similar reviews related to formula changes in the past. Unfortunately, I now need to find a new food that my cats will eat. 0.8131137971742012
Title: Yay Barley; Content: Right now I'm mostly just sprouting this so my cats can eat the grass. They love it. I rotate it around with Wheatgrass and Rye too 0.7912491093941975
Title: Twizzlers; Content: I love this candy. After weight watchers I had to cut back but still have a craving for it. 0.7796376922492927
Title: Good Quality Dog Food; Content: I have bought several of the Vitality canned dog food products and have found them all to be of good quality. The product looks more like a stew than a processed meat and it smells better. My Labrador is finicky and she appreciates this product better than most. 0.7745255229074056
Yep. That top result is a great match.
And once again: the exact phrase isn't in the text. But the concept is clearly there.
Wrapping It Up
Well done. You've taken a huge leap in your AI journey with Java and OpenAI.
Now it's time to play. Get some different content and grab the associated Embeddings. Then, conduct your own searches.
Also, feel free to check out the code in your copious free time.
Have fun!
Photo by Hassan OUAJBIR: https://www.pexels.com/photo/man-holding-white-disc-cases-1524072/