Browse Source

Hydrus integration, better display of embedded messages

pull/46/head
coomdev 2 years ago
parent
commit
aa5570c3d2
  1. 39
      README.md
  2. 2
      main.meta.js
  3. 26198
      main.user.js
  4. 59
      src/Components/App.svelte
  5. 5
      src/Components/Embedding.svelte
  6. 129
      src/Components/HydrusSearch.svelte
  7. 168
      src/Components/PostOptions.svelte
  8. 2
      src/Components/Tab.svelte
  9. 1
      src/Components/Tabs.svelte
  10. 14
      src/Components/Tag.svelte
  11. 3
      src/gif.ts
  12. 100
      src/hydrus.ts
  13. 4
      src/jpg.ts
  14. 68
      src/main.ts
  15. 5
      src/pngv3.ts
  16. 8
      src/stores.ts
  17. 54
      src/utils.ts
  18. 3
      src/webm.ts

39
README.md

@ -1,8 +1,6 @@
PNG Extra Embedder (PEE) PNG Extra Embedder (PEE)
======================== ========================
❗🔴🔥⚠️**Aussies: catbox is being censored by your ISP/Government, switch DNS or hang your politicians**⚠️🔥🔴❗
*Subsequently 'lolipiss' (**LOL** **I** **p**Want **i**To **s**Kill **s**Jannies)* *Subsequently 'lolipiss' (**LOL** **I** **p**Want **i**To **s**Kill **s**Jannies)*
Can embed any file in a PNG/WebM/GIF and upload it to a third-party host through 4chan. Can embed any file in a PNG/WebM/GIF and upload it to a third-party host through 4chan.
@ -18,6 +16,8 @@ How to Install
- [Install 4chanX (recommended)](https://www.4chan-x.net/builds/4chan-X.user.js) - [Install 4chanX (recommended)](https://www.4chan-x.net/builds/4chan-X.user.js)
- Use the prebuilt [main.user.js](https://git.coom.tech/coomdev/PEE/raw/branch/%E4%B8%AD%E5%87%BA%E3%81%97/main.user.js) - Use the prebuilt [main.user.js](https://git.coom.tech/coomdev/PEE/raw/branch/%E4%B8%AD%E5%87%BA%E3%81%97/main.user.js)
Note: 4chanX isn't a hard requirement, just recommended because it's overall a nicer experience. If you don't want to use 4chanX, make sure the native 4chan extension is enabled in your settings.
How to Build How to Build
============ ============
@ -36,12 +36,21 @@ this screenshot is outdated, UI has changed a little but I'm too lazy.
In the quick reply form, a magnet icon will appear. In the quick reply form, a magnet icon will appear.
Clicking it will allow you to add files to attach to the file that will be uploaded and shown on 4chan. Clicking it will allow you to add files to attach to the file that will be uploaded and shown on 4chan.
Hovering on the magnet will reveal a pencil icon, that will attach the content of your message box to the file, use it as a way to hide messages. And finally a checkmark that will do the uploading, make sure to have selected the file you'll post on 4chan beforehand. Hovering on the magnet will reveal a pencil icon, that will attach the content of your message box to the file, use it as a way to hide messages.
Your embeds will be attached as you add them after you've selected a file, but can be prepared before selecting your main file.
![qr](screen.png) ![qr](screen.png)
By default, you can add up to 5 attachments to a file. This limit can be raised, but keep in mind others using the default settings will only see your 5 first files, unless they themselves raised that limit in the settings. By default, you can add up to 5 attachments to a file. This limit can be raised, but keep in mind others using the default settings will only see your 5 first files, unless they themselves raised that limit in the settings.
### Thread Watcher
The "thread watcher" allows you to find threads that contain embeds.
A lot of the results might be false positives from people posting directly files from boorus, so you can adjust the perceptual hash filter settings to reduce that. Setting it to a very high value ensures results will be exclusively made of direct link embeds.
The "Contribute" checkbox makes your browser report posts with embeds you come accross during your regular browsing to [telepee](https://git.coom.tech/coomdev/telepee). It is recommended to enable it if you frequently post as it'll make your posts more visible to other extension users.
# <a id="coom"></a> TroubleShooting # <a id="coom"></a> TroubleShooting
## It doesn't work ## It doesn't work
@ -105,7 +114,7 @@ Just be discreet about it and you won't get into trouble.
Their OPs are wrongfully being banned under the pretense of using proxies/VPNs, or evading bans that didn't exist in the first place. Their OPs are wrongfully being banned under the pretense of using proxies/VPNs, or evading bans that didn't exist in the first place.
## Supports # Supports
Third Eye Third Eye
--------- ---------
@ -120,4 +129,24 @@ Supports:
- Base64 filenames - Base64 filenames
- [\<host>=\<file>] filenames - [\<host>=\<file>] filenames
- [\<type>=\<URL>] filenames (URL must be one of the supported hosts (catbox, pomf, zzzz...)) - [\<type>=\<URL>] filenames (URL must be one of the supported hosts (catbox, pomf, zzzz...))
- <6char file>(.\<ext>) filenames * \<type> is ignored and is inferred from the file content
Hydrus
------
By setting an API Key, you can automatically embed random files (prefiltered by your tags) into your uploads. You can also directly search, pick and embed from your Hydrus database from within PEE.
To generate an API Key, first enable the Hydrus Client API:
- Services > Manage Services > Client API
Leave the default port at 45869, enable CORS headers (required), and disable "allow non-local connections" (optional, but better security)
Apply your changes, then:
- Services > Review Services > Local > Client API > Add > Manually
Take note of the Access Key, enable the "Search for files" permission, apply your changes.
Then give this Access key to PEE where it's asked for.

2
main.meta.js

@ -1,7 +1,7 @@
// ==UserScript== // ==UserScript==
// @name PNGExtraEmbed // @name PNGExtraEmbed
// @namespace https://coom.tech/ // @namespace https://coom.tech/
// @version 0.183 // @version 0.184
// @description uhh // @description uhh
// @author You // @author You
// @match https://boards.4channel.org/* // @match https://boards.4channel.org/*

26198
main.user.js

File diff suppressed because it is too large

59
src/Components/App.svelte

@ -9,9 +9,9 @@
import Tab from "./Tab.svelte"; import Tab from "./Tab.svelte";
import TabPanel from "./TabPanel.svelte"; import TabPanel from "./TabPanel.svelte";
import { settings } from "../stores"; import { settings, appState } from "../stores";
import { filehosts } from "../filehosts"; import { filehosts } from "../filehosts";
import { appState } from "../../dist/stores"; import HydrusSearch from "./HydrusSearch.svelte";
let newbooru: Partial<Omit<Booru, "quirks"> & { view: string }> = {}; let newbooru: Partial<Omit<Booru, "quirks"> & { view: string }> = {};
let dial: Dialog; let dial: Dialog;
@ -78,6 +78,9 @@
<Tab>External</Tab> <Tab>External</Tab>
<Tab>File Host</Tab> <Tab>File Host</Tab>
<Tab>Thread Watcher</Tab> <Tab>Thread Watcher</Tab>
{#if $appState.akValid}
<Tab>Hydrus</Tab>
{/if}
</TabList> </TabList>
<TabPanel> <TabPanel>
<label> <label>
@ -138,6 +141,44 @@
title="You might still want to enable 'preload external files'">?</a title="You might still want to enable 'preload external files'">?</a
> >
</label> </label>
<label>
<input type="checkbox" bind:checked={$settings.hyd} />
<!-- svelte-ignore a11y-missing-attribute -->
Enable Hydrus Integration
</label>
{#if $settings.hyd}
{#if $appState.herror}
<span class="error">{$appState.herror}</span>
{/if}
<label>
Hydrus Access Key
<!-- svelte-ignore a11y-missing-attribute -->
<a
title="Only requires Search Files permission. See Hydrus docs on where to set this up."
>?</a
>
<input type="text" bind:value={$settings.ak} />
</label>
{#if $appState.akValid}
<label>
Auto-embed <input
style="width: 5ch;"
type="number"
bind:value={$settings.auto_embed}
/>
random files
<!-- svelte-ignore a11y-missing-attribute -->
</label>
<label>
<!-- svelte-ignore a11y-missing-attribute -->
<input
placeholder="Restrict to these tags (space to separate tags, _ to separate words)"
type="text"
bind:value={$settings.auto_tags}
/>
</label>
{/if}
{/if}
</TabPanel> </TabPanel>
<TabPanel> <TabPanel>
<label> <label>
@ -281,6 +322,11 @@
<p>Loading...</p> <p>Loading...</p>
{/if} {/if}
</TabPanel> </TabPanel>
{#if $appState.akValid}
<TabPanel>
<HydrusSearch />
</TabPanel>
{/if}
</Tabs> </Tabs>
</div> </div>
</div> </div>
@ -298,6 +344,11 @@
flex-wrap: wrap; flex-wrap: wrap;
} }
label > input[type="text"],
label > input[type="number"] {
width: 95%;
}
.enabled { .enabled {
display: block; display: block;
} }
@ -311,6 +362,10 @@
flex-direction: column; flex-direction: column;
} }
.error {
color: red;
}
hr { hr {
width: 100%; width: 100%;
} }

5
src/Components/Embedding.svelte

@ -10,6 +10,7 @@
export const dispatch = createEventDispatcher(); export const dispatch = createEventDispatcher();
export let file: EmbeddedFile; export let file: EmbeddedFile;
let isVideo = false; let isVideo = false;
let isImage = false; let isImage = false;
let isAudio = false; let isAudio = false;
@ -163,7 +164,11 @@
); );
} }
export let inhibitExpand: boolean = false;
export async function bepis(ev: MouseEvent) { export async function bepis(ev: MouseEvent) {
dispatch("click");
if (inhibitExpand) return;
if ($appState.isCatalog) return; if ($appState.isCatalog) return;
if (ev.button == 0) { if (ev.button == 0) {

129
src/Components/HydrusSearch.svelte

@ -0,0 +1,129 @@
<script lang="ts">
import { map } from "lodash";
import { each, onMount } from "svelte/internal";
import type { EmbeddedFile } from "../main";
import { appState } from "../stores";
import { addToEmbeds, getFileFromHydrus } from "../utils";
import Embedding from "./Embedding.svelte";
import Tag from "./Tag.svelte";
let tags: string[] = [];
let loading = false;
function removeTag(t: string) {
tags = tags.filter((e) => e != t);
update();
}
let maps: [number, EmbeddedFile][] = [];
async function update() {
loading = true;
if ($appState.client) {
try {
if (tags.length == 0) {
maps = [];
loading = false;
return;
}
maps = await getFileFromHydrus(
$appState.client,
tags.concat(["system:limit=32"]),
{ file_sort_type: 4 }
);
} catch {}
}
loading = false;
}
onMount(() => {
return update();
});
</script>
<div class="cont">
<input
type="text"
placeholder="Input a tag here, then press enter"
on:keydown={(ev) => {
if (ev.key == "Enter") {
if (ev.currentTarget.value)
tags = [...tags, ev.currentTarget.value];
ev.currentTarget.value = "";
update();
}
}}
/>
<details>
<summary>Tips</summary>
Press enter without entering a tag to refresh. <br />
Files are picked randomly <br />
Click on a file to embed it <br />
</details>
<div class="tagcont">
{#each tags as tag}
<Tag {tag} on:toggle={() => removeTag(tag)} />
{/each}
</div>
{#if loading}
Loading...
{:else}
<div class="results">
{#each maps as map (map[0])}
<Embedding
on:click={() => addToEmbeds(map[1])}
inhibitExpand={true}
id={"only"}
file={map[1]}
/>
{/each}
</div>
{/if}
</div>
<style scoped>
.results {
display: flex;
flex-wrap: wrap;
max-height: 30vh;
gap: 10px;
overflow-y: auto;
align-items: center;
justify-content: center;
}
.tagcont {
display: flex;
gap: 5px;
}
.cont {
display: flex;
flex-direction: column;
gap: 10px;
}
details {
border: 1px solid #aaa;
border-radius: 4px;
padding: 0.5em 0.5em 0;
}
summary {
font-weight: bold;
margin: -0.5em -0.5em 0;
padding: 0.5em;
cursor: pointer;
}
details[open] {
padding: 0.5em;
}
details[open] summary {
border-bottom: 1px solid #aaa;
margin-bottom: 0.5em;
}
</style>

168
src/Components/PostOptions.svelte

@ -1,84 +1,137 @@
<script lang="ts"> <script lang="ts">
import { appState, settings } from '../stores' import { appState, settings } from "../stores";
import type { ImageProcessor } from '../main' import type { ImageProcessor } from "../main";
import { fireNotification, getSelectedFile } from '../utils' import {
addToEmbeds,
embeddedToBlob,
fireNotification,
getFileFromHydrus,
uploadFiles,
} from "../utils";
export let processors: ImageProcessor[] = [] export let processors: ImageProcessor[] = [];
export let textinput: HTMLTextAreaElement export let textinput: HTMLTextAreaElement;
export let files: File[] = [] export let links: string[] = [];
const addContent = (...newfiles: File[]) => { const addContent = async (...newfiles: File[]) => {
files = [...files, ...newfiles] links = [...links, ...(await uploadFiles(newfiles))];
if (files.length > $settings.maxe) { return embedContent({} as any);
fireNotification( };
'warning',
`Can only add up to ${$settings.maxe} attachments, further attachments will be dropped`, let original: File | undefined;
) let currentEmbed: { file: Blob; name: string } | undefined;
files = files.slice(0, $settings.maxe)
} function restore() {
document.dispatchEvent(
new CustomEvent("QRSetFile", {
detail: { file: original },
})
);
} }
// This is an event to signal a change in the container file
document.addEventListener("PEEFile", async (e) => {
let file = (e as any).detail as File;
if (currentEmbed?.file != file) {
original = file;
if ($settings.auto_embed && $appState.client) {
const tags = $settings.auto_tags
.split(" ")
.map((e) => e.replaceAll("_", " "));
const efs = await getFileFromHydrus(
$appState.client,
tags.concat(["system:limit=" + $settings.auto_embed]),
{ file_sort_type: 4 }
);
const files = await embeddedToBlob(...efs.map(e => e[1]));
const nlinks = await uploadFiles(files);
links = [...links, ...nlinks];
}
embedContent(e);
}
});
document.addEventListener("QRPostSuccessful", () => {
if (currentEmbed) {
links = []; // cleanup
currentEmbed = undefined;
original = undefined;
}
});
document.addEventListener("AddPEE", (e) => {
let link = (e as any).detail as string | string[];
links = links.concat(link);
embedContent(e);
});
const embedText = async (e: Event) => { const embedText = async (e: Event) => {
if (textinput.value == '') return if (textinput.value == "") return;
if (textinput.value.length > 2000) { if (textinput.value.length > 2000) {
fireNotification("error", "Message attachments are limited to 2000 characters") fireNotification(
"error",
"Message attachments are limited to 2000 characters"
);
return; return;
} }
addContent( await addContent(
new File( new File(
[new Blob([textinput.value], { type: 'text/plain' })], [new Blob([textinput.value], { type: "text/plain" })],
`message${files.length}.txt`, `message${links.length}.txt`
), )
) );
textinput.value = '' textinput.value = "";
} };
const embedContent = async (e: Event) => { const embedContent = async (e: Event) => {
const file = await getSelectedFile() const file = original;
if (!file) return if (!file) return;
const type = file.type if (links.length == 0) return;
const type = file.type;
try { try {
const proc = processors const proc = processors
.filter((e) => e.inject) .filter((e) => e.inject)
.find((e) => e.match(file.name)) .find((e) => e.match(file.name));
if (!proc) throw new Error('Container filetype not supported') if (!proc) throw new Error("Container filetype not supported");
const buff = await proc.inject!(file, [...files].slice(0, $settings.maxe)) const buff = await proc.inject!(file, links.slice(0, $settings.maxe));
currentEmbed = {
file: new Blob([buff], { type }),
name: file.name,
} as unknown as { file: Blob; name: string };
document.dispatchEvent( document.dispatchEvent(
new CustomEvent('QRSetFile', { new CustomEvent("QRSetFile", {
//detail: { file: new Blob([buff]), name: file.name, type: file.type } detail: currentEmbed,
detail: { file: new Blob([buff], { type }), name: file.name }, })
}), );
)
files = [];
fireNotification( fireNotification(
'success', "success",
`File${files.length > 1 ? 's' : ''} successfully embedded!`, `File${links.length > 1 ? "s" : ""} successfully embedded!`
) );
} catch (err) { } catch (err) {
const e = err as Error const e = err as Error;
fireNotification('error', "Couldn't embed file: " + e.message) fireNotification("error", "Couldn't embed file: " + e.message);
} }
} };
const embedFile = async (e: Event) => { const embedFile = async (e: Event) => {
const input = document.createElement('input') as HTMLInputElement const input = document.createElement("input") as HTMLInputElement;
input.setAttribute('type', 'file') input.setAttribute("type", "file");
input.multiple = true input.multiple = true;
input.onchange = async (ev) => { input.onchange = async (ev) => {
if (input.files) { if (input.files) {
addContent(...input.files) addContent(...input.files);
} }
} };
input.click() input.click();
} };
</script> </script>
<div class="root"> <div class="root">
<!-- svelte-ignore a11y-missing-attribute --> <!-- svelte-ignore a11y-missing-attribute -->
<a on:click={embedFile} title="Add a file"> <a on:click={embedFile} title="Add a file">
<i class="fa fa-magnet"> {$appState.is4chanX ? '' : '🧲'} </i> <i class="fa fa-magnet"> {$appState.is4chanX ? "" : "🧲"} </i>
</a> </a>
<div class="additionnal"> <div class="additionnal">
<!-- svelte-ignore a11y-missing-attribute --> <!-- svelte-ignore a11y-missing-attribute -->
@ -86,16 +139,15 @@
on:click={embedText} on:click={embedText}
title="Add a message (this uses the content of the comment text box)" title="Add a message (this uses the content of the comment text box)"
> >
<i class="fa fa-pencil"> {$appState.is4chanX ? '' : '🖉'} </i> <i class="fa fa-pencil"> {$appState.is4chanX ? "" : "🖉"} </i>
</a>
<!-- svelte-ignore a11y-missing-attribute -->
<a on:click={embedContent} title="Ready to Embed (Select a file before)">
<i class="fa fa-check"> {$appState.is4chanX ? '' : '✅'} </i>
</a> </a>
{#if files.length} {#if links.length}
<!-- svelte-ignore a11y-missing-attribute --> <!-- svelte-ignore a11y-missing-attribute -->
<a on:click={() => (files = [])} title="Discard ALL selected content"> <a
<i class="fa fa-times"> {$appState.is4chanX ? '' : '❌'} </i> on:click={() => ((links = []), restore())}
title="Discard ALL selected content"
>
<i class="fa fa-times"> {$appState.is4chanX ? "" : "❌"} </i>
</a> </a>
{/if} {/if}
</div> </div>

2
src/Components/Tab.svelte

@ -29,6 +29,6 @@
.selected { .selected {
border-bottom: 2px solid; border-bottom: 2px solid;
color: #333; color: #f6ff76;
} }
</style> </style>

1
src/Components/Tabs.svelte

@ -59,5 +59,6 @@
.tabs { .tabs {
display: flex; display: flex;
flex-direction: column; flex-direction: column;
gap: 5px;
} }
</style> </style>

14
src/Components/Tag.svelte

@ -1,23 +1,23 @@
<script lang="ts"> <script lang="ts">
import { createEventDispatcher } from 'svelte' import { createEventDispatcher } from "svelte";
export let tag: string export let tag: string;
export let toggleable = false export let toggleable = false;
export let toggled = false export let toggled = false;
const dispatch = createEventDispatcher() const dispatch = createEventDispatcher();
</script> </script>
<span <span
class:toggle={toggleable} class:toggle={toggleable}
class:toggled={toggleable && toggled} class:toggled={toggleable && toggled}
on:click={() => dispatch('toggle')} on:click={() => dispatch("toggle")}
class="tag" class="tag"
> >
{tag} {tag}
{#if toggleable} {#if toggleable}
<span on:click={e => (e.preventDefault(), dispatch('remove'))}>x</span> <span on:click={(e) => (e.preventDefault(), dispatch("remove"))}>x</span>
{/if} {/if}
</span> </span>

3
src/gif.ts

@ -86,11 +86,10 @@ const write_embedding = async (writer: WritableStreamDefaultWriter<Buffer>, inj:
} }
}; };
const inject = async (container: File, injs: File[]) => { const inject = async (container: File, links: string[]) => {
const [writestream, extract] = BufferWriteStream(); const [writestream, extract] = BufferWriteStream();
const writer = writestream.getWriter(); const writer = writestream.getWriter();
const links = await uploadFiles(injs);
const inj = Buffer.from(links.join(' ')); const inj = Buffer.from(links.join(' '));
const contbuff = Buffer.from(await container.arrayBuffer()); const contbuff = Buffer.from(await container.arrayBuffer());

100
src/hydrus.ts

@ -0,0 +1,100 @@
type TagList = (string | TagList)[];
export interface MyTags {
0: string[];
}
export interface AllKnownTags {
0: string[];
}
export interface ServiceNamesToStatusesToTags {
'all known tags': AllKnownTags;
}
export interface MyTags2 {
0: string[];
}
export interface AllKnownTags2 {
0: string[];
}
export interface ServiceNamesToStatusesToDisplayTags {
'all known tags': AllKnownTags2;
}
export interface Metadata {
file_id: number;
hash: string;
size: number;
mime: string;
ext: string;
width: number;
height: number;
duration?: any;
num_frames?: any;
num_words?: any;
has_audio: boolean;
time_modified: number;
is_inbox: boolean;
is_local: boolean;
is_trashed: boolean;
known_urls: string[];
service_names_to_statuses_to_tags: ServiceNamesToStatusesToTags;
service_names_to_statuses_to_display_tags: ServiceNamesToStatusesToDisplayTags;
}
export interface RootObject {
metadata: Metadata[];
}
export class HydrusClient {
constructor(
private ak: string,
private origin: string = 'http://127.0.0.1',
private port: number = 45869,
) {
}
get baseUrl() {
return `${this.origin}:${this.port}`;
}
async get(params: string) {
return await fetch(this.baseUrl + params, {
headers: {
'Hydrus-Client-API-Access-Key': this.ak
}
});
}
async verify() {
try {
const ret = await this.get('/verify_access_key');
return !!await ret.json();
} catch (e) {
return false;
}
}
async idsByTags(taglist: TagList, args?: object) {
const req = await this.get('/get_files/search_files?tags=' + encodeURIComponent(JSON.stringify(taglist)) + (args ? '&' + (Object.entries(args).map(e => `${e[0]}=${encodeURIComponent(e[1])}`).join('&')) : ''));
return await req.json() as { file_ids: number[] };
}
async getMetaDataByIds(ids: number[]) {
const req = await this.get('/get_files/file_metadata?file_ids=' + encodeURIComponent(JSON.stringify(ids)));
return await req.json() as { metadata: Metadata[] };
}
async getFile(id: number) {
const req = await this.get('/get_files/file?file_id=' + id);
return await req.arrayBuffer();
}
async getThumbnail(id: number) {
const req = await this.get('/get_files/thumbnail?file_id=' + id);
return await req.arrayBuffer();
}
}

4
src/jpg.ts

@ -46,12 +46,12 @@ export const convertToPng = async (f: File): Promise<Blob | undefined> => {
} }
}; };
const inject = async (b: File, c: File[]) => { const inject = async (b: File, links: string[]) => {
const pngfile = await convertToPng(b); const pngfile = await convertToPng(b);
if (!pngfile || pngfile.size > 3000 * 1024) { if (!pngfile || pngfile.size > 3000 * 1024) {
throw new Error("Couldn't convert file to PNG: resulting filesize too big."); throw new Error("Couldn't convert file to PNG: resulting filesize too big.");
} }
return pngv3.inject!(new File([pngfile], b.name), c); return pngv3.inject!(new File([pngfile], b.name), links);
}; };
export default { export default {

68
src/main.ts

@ -18,17 +18,18 @@ import SettingsButton from './Components/SettingsButton.svelte';
import Embeddings from './Components/Embeddings.svelte'; import Embeddings from './Components/Embeddings.svelte';
import EyeButton from './Components/EyeButton.svelte'; import EyeButton from './Components/EyeButton.svelte';
import NotificationsHandler from './Components/NotificationsHandler.svelte'; import NotificationsHandler from './Components/NotificationsHandler.svelte';
import { fireNotification } from "./utils"; import { fireNotification, getSelectedFile } from "./utils";
import { getQueryProcessor, QueryProcessor } from "./websites"; import { getQueryProcessor, QueryProcessor } from "./websites";
import { ifetch, streamRemote, supportedAltDomain } from "./platform"; import { ifetch, streamRemote, supportedAltDomain } from "./platform";
import TextEmbeddingsSvelte from "./Components/TextEmbeddings.svelte"; import TextEmbeddingsSvelte from "./Components/TextEmbeddings.svelte";
import { HydrusClient } from "./hydrus";
export interface ImageProcessor { export interface ImageProcessor {
skip?: true; skip?: true;
match(fn: string): boolean; match(fn: string): boolean;
has_embed(b: Buffer, fn?: string, prevurl?: string): boolean | Promise<boolean>; has_embed(b: Buffer, fn?: string, prevurl?: string): boolean | Promise<boolean>;
extract(b: Buffer, fn?: string): EmbeddedFile[] | Promise<EmbeddedFile[]>; extract(b: Buffer, fn?: string): EmbeddedFile[] | Promise<EmbeddedFile[]>;
inject?(b: File, c: File[]): Buffer | Promise<Buffer>; inject?(b: File, c: string[]): Buffer | Promise<Buffer>;
} }
let qp: QueryProcessor; let qp: QueryProcessor;
@ -37,11 +38,29 @@ let processors: ImageProcessor[] =
[thirdeye, pomf, pngv3, jpg, webm, gif]; [thirdeye, pomf, pngv3, jpg, webm, gif];
let cappState: Parameters<typeof appState['set']>[0]; let cappState: Parameters<typeof appState['set']>[0];
settings.subscribe(b => { settings.subscribe(async b => {
if (b.hyd) {
// transition from disable to enabled
if (b.ak) {
const hydCli = new HydrusClient(b.ak);
console.log(b.ak);
let herror: string | undefined;
try {
const valid = await hydCli.verify();
if (!valid)
herror = "Hydrus appears to not be running or the key is wrong.";
appState.set({ ...cappState, akValid: valid, client: hydCli, herror });
} catch {
herror = "Hydrus appears to not be running";
appState.set({ ...cappState, akValid: false, client: null, herror });
}
}
}
csettings = b; csettings = b;
processors = [...(!csettings.te ? [thirdeye] : []), processors = [...(!csettings.te ? [thirdeye] : []),
pngv3, pomf, jpg, webm, gif pngv3, pomf, jpg, webm, gif
]; ];
}); });
appState.subscribe(v => { appState.subscribe(v => {
@ -445,22 +464,6 @@ document.addEventListener('QRDialogCreation', <any>((e: CustomEvent<HTMLElement>
props: { processors, textinput: (e.detail || e.target).querySelector('textarea')! } props: { processors, textinput: (e.detail || e.target).querySelector('textarea')! }
}); });
const checkEvent = (e: Event) => {
if ((po as any).files.length > 0) {
e.preventDefault();
e.stopImmediatePropagation();
e.stopPropagation();
fireNotification("error", "You have files you forgot to embed!");
return false;
}
};
document.addEventListener("keydown", (e) => {
if (e.ctrlKey && (e.key == "Enter" || e.keyCode == 13)) {
return checkEvent(e);
}
}, true);
let target; let target;
if (!cappState.is4chanX) { if (!cappState.is4chanX) {
target = e.detail; target = e.detail;
@ -470,23 +473,17 @@ document.addEventListener('QRDialogCreation', <any>((e: CustomEvent<HTMLElement>
else { else {
target = e.target as HTMLDivElement; target = e.target as HTMLDivElement;
target.querySelector('#qr-filename-container')?.appendChild(a); target.querySelector('#qr-filename-container')?.appendChild(a);
const sub = target.querySelector("input[type=submit]") as HTMLElement; const filesinp = target.querySelector('#file-n-submit') as HTMLInputElement;
let prevFile: File;
sub.addEventListener("click", checkEvent, true); const obs = new MutationObserver(async (m) => {
// file possibly changed
const obs = new MutationObserver((m) => { const currentFile = await getSelectedFile();
for (const r of m) { if (prevFile != currentFile) {
switch (r.type) { prevFile = currentFile;
case "attributes": document.dispatchEvent(new CustomEvent("PEEFile", { detail: prevFile }));
break;
case "characterData":
break;
}
} }
}); });
obs.observe(sub, { obs.observe(filesinp, { attributes: true });
attributes: true
});
} }
}), { once: !cappState!.is4chanX }); // 4chan's normal extension destroys the QR form everytime }), { once: !cappState!.is4chanX }); // 4chan's normal extension destroys the QR form everytime
@ -549,8 +546,7 @@ function processAttachments(post: HTMLDivElement, ress: [EmbeddedFile, boolean][
props: { props: {
files: ress.map(e => e[0]).filter(e => files: ress.map(e => e[0]).filter(e =>
Buffer.isBuffer(e.data) && e.filename.endsWith('.txt') && e.filename.startsWith('message') Buffer.isBuffer(e.data) && e.filename.endsWith('.txt') && e.filename.startsWith('message')
), )
id: '' + id
} }
}); });
const emb = new Embeddings({ const emb = new Embeddings({

5
src/pngv3.ts

@ -1,7 +1,7 @@
import { Buffer } from "buffer"; import { Buffer } from "buffer";
import type { EmbeddedFile, ImageProcessor } from "./main"; import type { EmbeddedFile, ImageProcessor } from "./main";
import { PNGDecoder, PNGEncoder } from "./png"; import { PNGDecoder, PNGEncoder } from "./png";
import { buildPeeFile, decodeCoom3Payload, fireNotification, uploadFiles } from "./utils"; import { decodeCoom3Payload } from "./utils";
const CUM3 = Buffer.from("doo\0" + "m"); const CUM3 = Buffer.from("doo\0" + "m");
@ -88,8 +88,7 @@ export const inject_data = async (container: File, injb: Buffer) => {
}; };
const inject = async (container: File, injs: File[]) => { const inject = async (container: File, links: string[]) => {
const links = await uploadFiles(injs);
const injb = Buffer.from(links.join(' ')); const injb = Buffer.from(links.join(' '));
return inject_data(container, injb); return inject_data(container, injb);
}; };

8
src/stores.ts

@ -1,4 +1,5 @@
import { writable } from "svelte/store"; import { writable } from "svelte/store";
import type { HydrusClient } from "./hydrus";
import type { Booru } from "./thirdeye"; import type { Booru } from "./thirdeye";
export const localLoad = <T>(key: string, def: T) => export const localLoad = <T>(key: string, def: T) =>
@ -15,6 +16,10 @@ export const settings = writable(localLoad('settingsv2', {
dh: false, dh: false,
xpv: false, xpv: false,
xpi: false, xpi: false,
hyd: false,
ak: '',
auto_embed: 0,
auto_tags: '',
te: false, te: false,
eye: false, eye: false,
ca: false, ca: false,
@ -82,6 +87,9 @@ export const settings = writable(localLoad('settingsv2', {
export const appState = writable({ export const appState = writable({
isCatalog: false, isCatalog: false,
is4chanX: false, is4chanX: false,
akValid: false,
herror: '' as string | undefined,
client: null as HydrusClient | null,
foundPosts: [] as HTMLElement[] foundPosts: [] as HTMLElement[]
}); });

54
src/utils.ts

@ -4,6 +4,9 @@ import type { EmbeddedFile } from './main';
import { settings } from "./stores"; import { settings } from "./stores";
import { filehosts } from "./filehosts"; import { filehosts } from "./filehosts";
import { getHeaders, ifetch, Platform } from "./platform"; import { getHeaders, ifetch, Platform } from "./platform";
import type { HydrusClient } from "./hydrus";
import { GM_fetch } from "./requests";
import { fileTypeFromBuffer } from "file-type";
export let csettings: Parameters<typeof settings['set']>[0]; export let csettings: Parameters<typeof settings['set']>[0];
@ -58,7 +61,7 @@ const generateThumbnail = async (f: File): Promise<Buffer> => {
const blob = await new Promise<Blob | null>(_ => can.toBlob(_, "image/jpg")); const blob = await new Promise<Blob | null>(_ => can.toBlob(_, "image/jpg"));
if (!blob) if (!blob)
return Buffer.alloc(0); return Buffer.alloc(0);
return new Buffer(await blob.arrayBuffer()); return Buffer.from(await blob.arrayBuffer());
}; };
export const buildPeeFile = async (f: File) => { export const buildPeeFile = async (f: File) => {
@ -83,7 +86,7 @@ export const buildPeeFile = async (f: File) => {
thumbnail.copy(ret, ptr); thumbnail.copy(ret, ptr);
ptr += thumbnail.byteLength; ptr += thumbnail.byteLength;
} }
new Buffer(await f.arrayBuffer()).copy(ret, ptr); Buffer.from(await f.arrayBuffer()).copy(ret, ptr);
return new Blob([ret]); return new Blob([ret]);
}; };
@ -210,3 +213,50 @@ export const getSelectedFile = () => {
document.dispatchEvent(new CustomEvent('QRGetFile')); document.dispatchEvent(new CustomEvent('QRGetFile'));
}); });
}; };
export async function embeddedToBlob(...efs: EmbeddedFile[]) {
return (await Promise.all(efs.map(async ef => {
let buff: Buffer;
if (typeof ef.data == "string") {
const req = await GM_fetch(ef.data);
buff = Buffer.from(await req.arrayBuffer());
} else if (!Buffer.isBuffer(ef.data))
buff = await ef.data();
else
buff = ef.data;
const mim = await fileTypeFromBuffer(buff);
const file = new File([buff], ef.filename, { type: mim?.mime });
return file;
}))).filter(e => e);
}
export async function addToEmbeds(...efs: EmbeddedFile[]) {
const files = await embeddedToBlob(...efs);
const links = await uploadFiles(files);
document.dispatchEvent(new CustomEvent("AddPEE", { detail: links }));
}
export async function getFileFromHydrus(client: HydrusClient,
tags: string[], args?: any) {
const results = (
await client.idsByTags(tags, args)
).file_ids;
const metas = await client.getMetaDataByIds(results);
return await Promise.all(
results.map(async (id, idx) => {
return [
id,
{
thumbnail: Buffer.from(
await client.getThumbnail(id)!
),
data: async () =>
Buffer.from(
await client.getFile(id)!
),
filename: 'file' + metas.metadata[idx].ext,
},
] as [number, EmbeddedFile];
})
);
}

3
src/webm.ts

@ -124,8 +124,7 @@ const extract = (webm: Buffer) => {
return decodeCoom3Payload(chk.data); return decodeCoom3Payload(chk.data);
}; };
const inject = async (container: File, injs: File[]): Promise<Buffer> => { const inject = async (container: File, links: string[]): Promise<Buffer> => {
const links = await uploadFiles(injs);
return embed(Buffer.from(await container.arrayBuffer()), Buffer.from(links.join(' '))); return embed(Buffer.from(await container.arrayBuffer()), Buffer.from(links.join(' ')));
}; };

Loading…
Cancel
Save