I just managed to solve the most annoying problem with implementing this feature, which is how to successfully run the queries in a separate AppDomain without serializing all the data:
Am I understanding this right -- you're "serializing" to the database instead of across app domains? If so, then how is this faster? Aren't you trading memory access for network access?
I'm curious, what use-case does an ad-hoc query have? Are these designed to be used e.g., by an administrator inspecting data in the document store, who wants to be able to query and filter the data? Or, are these actually intended for use by client code?
@Nathan, RavenDB compiles queries dynamically, but the runtime compiles them to a new assembly. Since unloading assemblies at run time isn't supported, those assemblies generated from the queries will stick around for the lifetime of the server instance - that's a memory leak. Unloading entire AppDomains IS supported, though - so the only way to avoid this memory leak is to load the assemblies in a new AppDomain, and then unload it when it's no longer needed.
I believe using Reflection.Emit instead of Expression.Compile would also avoid it (because you're creating a DynamicMethod instance, not a new assembly), but that would require writing a library to generate IL for arbitrary expression trees, which is a potentially arduous undertaking.
> Email-style angle brackets
> are used for blockquotes.
> > And, they can be nested.
> #### Headers in blockquotes
>
> * You can quote a list.
> * Etc.
Horizontal Rules
Three or more dashes or asterisks:
---
* * *
- - - -
Manual Line Breaks
End a line with two or more spaces:
Roses are red,
Violets are blue.
Fenced Code Blocks
Code blocks delimited by 3 or more backticks or tildas:
```
This is a preformatted
code block
```
Header IDs
Set the id of headings with {#<id>} at end of heading line:
## My Heading {#myheading}
Tables
Fruit |Color
---------|----------
Apples |Red
Pears |Green
Bananas |Yellow
Definition Lists
Term 1
: Definition 1
Term 2
: Definition 2
Footnotes
Body text with a footnote [^1]
[^1]: Footnote text here
Abbreviations
MDD <- will have title
*[MDD]: MarkdownDeep
FUTURE POSTS
RavenDB 7.1: Next-Gen Pagers - 5 days from now
RavenDB 7.1: Write modes - 7 days from now
RavenDB 7.1: Reclaiming disk space - 9 days from now
RavenDB 7.1: Shared Journals - 12 days from now
There are posts all the way to Feb 17, 2025
RECENT SERIES
Challenge
(77): 03 Feb 2025 - Giving file system developer ulcer
Answer
(13): 22 Jan 2025 - What does this code do?
Comments
Wow, what a code. Looks like you have yet to work on ad-hoc query feature.
Just a question, why would I want to run a query from a different AppDomain?
Am I understanding this right -- you're "serializing" to the database instead of across app domains? If so, then how is this faster? Aren't you trading memory access for network access?
Why not simply use MarshalByRefObjects to contain the data that would have been serialized?
I'm curious, what use-case does an ad-hoc query have? Are these designed to be used e.g., by an administrator inspecting data in the document store, who wants to be able to query and filter the data? Or, are these actually intended for use by client code?
Oh, you are using them - the QueryRunner is one. Am I right here?
Can you please elaborate? It is clear that there was a query from a different AppDomain, but it is not clear what is actually serialized.
Also please explain if this is a feature or just an optimization for the different AppDomain use case.
Thanks.
@Nathan, RavenDB compiles queries dynamically, but the runtime compiles them to a new assembly. Since unloading assemblies at run time isn't supported, those assemblies generated from the queries will stick around for the lifetime of the server instance - that's a memory leak. Unloading entire AppDomains IS supported, though - so the only way to avoid this memory leak is to load the assemblies in a new AppDomain, and then unload it when it's no longer needed.
I believe using Reflection.Emit instead of Expression.Compile would also avoid it (because you're creating a DynamicMethod instance, not a new assembly), but that would require writing a library to generate IL for arbitrary expression trees, which is a potentially arduous undertaking.
I LOVE your understatement
Comment preview