Database support

Topics: Developer Forum
Developer
Sep 10, 2006 at 2:02 PM
Could you describe the SharpEntropy tool's state of database support?
I really like the basic ideal of reading models from databases, but there are several things I'm concerned about:
-the choice of SQLite -in my experience, for really large datasets, it doesn't scale up so well -depending on how you solved the storage, it might be a problem. Have you done any measurements, like comparing the overall speed of sentence tagging from .nbin VS database? of course, if you used ADO.NET, then simply changing the database underneath it won't be a problem at all

-is it able to run statelessly from database, or does it have to pre-load the whole database before being able to use the models?
Coordinator
Sep 12, 2006 at 12:15 PM

> the choice of SQLite -in my experience, for really large datasets, >it doesn't scale up so well

It's not hard to create other readers/writers for other databases. I have one for SQL Server pretty much done. I used Sqlite because (a) it is file-based, so it was easier to change clients to test it (b) I was using Sqlite for something else at the time.

> depending on how you solved the storage, it might be a problem. Have > you done any measurements, like comparing the overall speed of > sentence tagging from .nbin VS database?

I have done some performance testing, but my machine can hold all the .nbin data in memory, so .nbin is always orders of magnitude faster than a solution that reads from disk as necessary rather than reading everything up front.

> is it able to run statelessly from database
It reads in the outcome names up front (which is trivial), but other than that, data is accessed from the database as required rather than being read all at once up-front as it is with the .nbin models.

I changed the class organisation of SharpEntropy quite a bit to enable this to happen, which is one reason wny SharpEntropy's structure has deviated quite a bit from the Java MaxEnt library.

Jan 29, 2008 at 5:51 PM
Hi,

Not sure if this is an issue, but I could not run the test.exe program in .\SQLite.NET\bin directory. The System.Data.SQLite.dll seems not to build right - the System.Data.SQLite.dll taken from SQLite website works properly with this test.exe file.

Second thing I want to ask is: where can I find example(s) of the utilization of SQL translation within SharpNLP? What/How is the NLP model presentation created?

Thank you very much.
Aug 27, 2008 at 12:07 PM
Hi,
If this thread is alive then can somebody help me in using SqliteGisModelWriter and SqliteGisModelReader. I am continuously getting following exception:

Unable to load DLL 'SQLite.Interop.DLL': The specified module could not be found. (Exception from HRESULT: 0x8007007E)
From where I can find this dll? or what are the alternate ways to resolve this issue?

Thanks in advance.