Spedon commited on
Commit
791e506
·
verified ·
1 Parent(s): e83f4b0

docs: enhance `texify` model documentation and usage examples

Browse files

- Add a base model reference to `vikp/texify`
- Include a pipeline tag for `image-to-text`
- Provide detailed usage instructions for the `texify-fp16-onnx` model
- Add an example Python code snippet for using the model with `optimum[onnxruntime]`
- Include input and output image examples for visualization

Signed-off-by: Spedon Wen <[email protected]>

Files changed (1) hide show
  1. README.md +40 -0
README.md CHANGED
@@ -1,3 +1,43 @@
1
  ---
2
  license: cc-by-sa-4.0
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: cc-by-sa-4.0
3
+ base_model:
4
+ - vikp/texify
5
+ pipeline_tag: image-to-text
6
  ---
7
+
8
+ ## texify-fp16-onnx
9
+
10
+ https://huggingface.co/vikp/texify with fp16 ONNX weights, shoutout to https://huggingface.co/Xenova/texify
11
+
12
+ ## Usage (`optimum[onnxruntime]`)
13
+
14
+ If you haven't already, you can install the optimum with the onnxrumtime backend
15
+
16
+ ```bash
17
+ pip install "optimum[onnxruntime-gpu]"
18
+ ```
19
+
20
+ **Example:**
21
+
22
+ ```python
23
+ from optimum.onnxruntime import ORTModelForVision2Seq
24
+ from optimum.pipelines import pipeline
25
+
26
+ model = ORTModelForVision2Seq.from_pretrained("Spedon/texify-fp16-onnx", provider="CUDAExecutionProvider")
27
+ texify = pipeline(
28
+ "image-to-text",
29
+ model,
30
+ feature_extractor="Spedon/texify-fp16-onnx",
31
+ image_processor="Spedon/texify-fp16-onnx",
32
+ )
33
+ image = (
34
+ "https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/latex.png"
35
+ )
36
+ latex = texify(image, max_new_tokens=384)
37
+ print(latex)
38
+ # [{'generated_text': "The potential $V_i$ of cell $\\mathcal{C}_i$ centred at position $\\mathbf{r}_i$ is related to the surface charge densities $\\sigma_j$ of cells $\\mathcal{C}_j$ $j\\in[1,N]$ through the superposition principle as: $$V_i\\,=\\,\\sum_{j=0}^{N}\\,\\frac{\\sigma_j}{4\\pi\\varepsilon_0}\\,\\int_{\\mathcal{C}_j}\\frac{1}{\\|\\mathbf{r}_i-\\mathbf{r}'\\|}\\,\\mathrm{d}^2\\mathbf{r}'\\,=\\,\\sum_{j=0}^{N}\\,Q_{ij}\\,\\sigma_j,$$ where the integral over the surface of cell $\\mathcal{C}_j$ only depends on $\\mathcal{C}_j$ shape and on the relative position of the target point $\\mathbf{r}_i$ with respect to $\\mathcal{C}_j$ location, as $\\sigma_j$ is assumed constant over the whole surface of cell $\\mathcal{C}_j$. "}]
39
+ ```
40
+
41
+ | Input image | Visualized output |
42
+ | ---------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
43
+ | ![image/png](https://cdn-uploads.huggingface.co/production/uploads/61b253b7ac5ecaae3d1efe0c/9UNWPwjFM-dRVf6m1gYJV.png) | ![image/png](https://cdn-uploads.huggingface.co/production/uploads/61b253b7ac5ecaae3d1efe0c/BK4wkPTqqvlTYeTPeEXTh.png) |