activescott's Notes

Cool laptops and desktops for Linux.

You can use the OpenAI Compatible Provider package to use language model providers that implement the OpenAI API.

Below we focus on the general setup and provider instance creation. You can also write a custom provider package leveraging the OpenAI Compatible package.

We provide detailed documentation for the following OpenAI compatible providers:

LM Studio
NIM
Heroku
Clarifai

The general setup and provider instance creation is the same for all of these providers.

HLS.js is a JavaScript library that implements an HTTP Live Streaming client. It relies on HTML5 video and MediaSource Extensions for playback.

It works by transmuxing MPEG-2 Transport Stream and AAC/MP3 streams into ISO BMFF (MP4) fragments. Transmuxing is performed asynchronously using a Web Worker when available in the browser. HLS.js also supports HLS + fmp4, as announced during WWDC2016.

HLS.js works directly on top of a standard HTML element.

detailed benchmarks and info wrt apple silicon cpus with llama.

LlamaBarn is a macOS menu bar app for running local LLMs.

While Vulkan can be a good fallback, for LLM inference at least, the performance difference is not as insignificant as you believe. I just ran a test on the latest pull just to make sure this is still the case on llama.cpp HEAD, but text generation is +44% faster and prompt processing is +202% (~3X) faster with ROCm vs Vulkan.

Note: if you're building llama.cpp, all you have to do is swap GGML_HIPBLAS=1 and GGML_VULKAN=1 so the extra effort is just installing ROCm? (vs the Vulkan devtools)

ROCm:

CUDA_VISIBLE_DEVICES=1 ./llama-bench -m /models/gguf/llama-2-7b.Q4_0.gguf ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no ggml_cuda_init: found 1 ROCm devices: Device 0: Radeon RX 7900 XTX, compute capability 11.0, VMM: no | model | size | params | backend | ngl | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | ------------: | -------------------: | | llama 7B Q4_0 | 3.56 GiB | 6.74 B | ROCm | 99 | pp512 | 3258.67 ± 29.23 | | llama 7B Q4_0 | 3.56 GiB | 6.74 B | ROCm | 99 | tg128 | 103.31 ± 0.03 |

build: 31ac5834 (3818)

Vulkan:

GGML_VK_VISIBLE_DEVICES=1 ./llama-bench -m /models/gguf/llama-2-7b.Q4_0.gguf | model | size | params | backend | ngl | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | ------------: | -------------------: | ggml_vulkan: Found 1 Vulkan devices: Vulkan0: Radeon RX 7900 XTX (RADV NAVI31) (radv) | uma: 0 | fp16: 1 | warp size: 64 | llama 7B Q4_0 | 3.56 GiB | 6.74 B | Vulkan | 99 | pp512 | 1077.49 ± 2.00 | | llama 7B Q4_0 | 3.56 GiB | 6.74 B | Vulkan | 99 | tg128 | 71.83 ± 0.06 |

build: 31ac583

This guide highlights the key features of the new SvelteKit-based WebUI of llama.cpp.

The new WebUI in combination with the advanced backend capabilities of the llama-server delivers the ultimate local AI chat experience. A few characteristics that set this project ahead of the alternatives:

Free, open-source and community-driven
Excellent performance on all hardware
Advanced context and prefix caching
Parallel and remote user support
Extremely lightweight and memory efficient
Vibrant and creative community
100% privacy

If you back up your files to Backblaze B2 using software that uploads files when they change, Backblaze B2 retains old versions of the file. This is helpful because the old versions can be used if the original file is accidentally deleted. On the other hand, keeping old versions forever can clutter things. For an application like this, you may want a Lifecycle Rule that keeps old versions in a backup/ folder for 30 days and then deletes them, as in the following example:

Keep only the last version of the file This rule keeps only the most current version of a file. The previous version of the file is "hidden" for one day and then deleted.

Cockpit is a web-based graphical interface for servers, intended for everyone, especially those who are:

Thanks to Cockpit intentionally using system APIs and commands, a whole team of admins can manage a system in the way they prefer, including the command line and utilities right alongside Cockpit.

Cockpit makes Linux discoverable. You don’t have to remember commands at a command-line.

See your server in a web browser and perform system tasks with a mouse. It’s easy to start containers, administer storage, configure networks, and inspect logs. Basically, you can think of Cockpit like a graphical “desktop interface”, but for individual servers.

Cockpit uses APIs that already exist on the system. It doesn’t reinvent subsystems or add a layer of its own tooling.

By default, Cockpit uses your system’s normal user logins and privileges. Network-wide logins are also supported through single-sign-on and other authentication techniques.

Cockpit itself doesn’t eat resources or even run in the background when you’re not using it. It runs on demand, thanks to systemd socket activation.

The US wants a clear and enforceable commitment that Iran will not develop nuclear weapons – or even the capability to do so quickly.

Washington and Tehran signed a nuclear deal in 2015 under US President Barack Obama. The agreement put a limit on Iran’s uranium enrichment of 3.67 percent in return for sanctions relief. But Trump, who succeeded Obama, withdrew Washington from the deal three years later and slapped sanctions back on Iran. Since then, Iran has accelerated its uranium enrichment to 60 percent. To make an atomic bomb, 90 percent enrichment is required.

During Israel’s 12-day war on Iran in June, the US carried out air strikes on Iran’s three main nuclear sites, after which Trump claimed that Iran’s nuclear programme had been obliterated. But eight months later, he started a war against Iran by saying one of his main goals was to keep Iran from obtaining a nuclear weapon.

The war was launched while talks mediated by Oman were under way between Iran and the US. Oman had said a short time before the attacks began that a deal was “within reach”.

Iran is pushing for a broader regional ceasefire, including an end to fighting involving its allies, such as Hezbollah in Lebanon.

While Israeli Prime Minister Benjamin Netanyahu has expressed support for Washington’s decision to suspend the strikes on Iran, he said the ceasefire will not extend to Israel’s ongoing military operations in Lebanon.

Hours into the ceasefire, which began on Wednesday, Israel carried out dozens of attacks across Lebanon, killing more than 300 people in one day.

However, Tehran insisted the ceasefire included Lebanon, citing Pakistani Prime Minister Shehbaz Sharif’s ceasefire announcement on X, which unequivocally stated this was the case.

patch intel macs to get them working with newer OS versions.

Ghostty is a fast, feature-rich, and cross-platform terminal emulator that uses platform-native UI and GPU acceleration.

The /terminal-setup command is only visible in terminals that require manual configuration. If you’re using iTerm2, WezTerm, Ghostty, or Kitty, you won’t see this command because Shift+Enter already works natively.

Looking for iMovie Old Versions? Here are the Downloads.

“Tuesday will be Power Plant Day, and Bridge Day, all wrapped up in one, in Iran. There will be nothing like it!!!” posted President Donald Trump on Easter Sunday. In case one thought that was an impulsive utterance, it’s notable that the president in apparently prepared remarks a few days earlier said, “If there is no deal, we are going to hit each and every one of their electric generating plants very hard and probably simultaneously.”

Such rhetorical statements – if followed through – would amount to the most serious war crimes – and thus the president’s statements place servicemembers in a profoundly challenging situation.

Iranian power plants and other critical civilian infrastructure are protected from attacks by the law of war the United States helped craft after World War II. Such an object can lose its protection only if it is used for military purposes by the enemy and its destruction “offers a definite military advantage.” Even then, such an object can be attacked only if, after a case-by-case rigorous analysis, the “concrete and direct military advantage anticipated” outweighs the civilian suffering that is expected to result. (Geneva Convention Additional Protocol I art. 52, art. 57; DOD Law of War Manual, § 5.6, § 5.12).

Despite those well-settled legal parameters, President Trump has repeatedly threatened to obliterate such infrastructure without regard to the law’s high demands. His comments are blatant expressions that he is willing to turn the United States into a rogue State like Iran and Russia, one that rejects the fundamental legal restraints that protect innocent non-combatants like children, and the Iranian civilian population itself.

U.S. military commanders have sworn to obey the Constitution and only those orders from their superiors that are lawful.  Threats to bomb Iran “back to the Stone Ages” and to show “no quarter, no mercy” are plainly illegal.  Trump’s outrageous statements gravely threaten our military professionals’ bedrock moral and legal principles, ones enshrined in the law of war that they’ve been trained to follow their entire careers.

the DOD Law of War Manual’s note on targeting civilian infrastructure states: “Diminishing the morale of the civilian population and their support for the war effort does not provide a definite military advantage. However, attacks that are otherwise lawful are not rendered unlawful if they happen to result in diminished civilian morale.”  DOD Law of War Manual, § 5.6.  Such “morale bombing” has been rejected for many decades; it had gained support during World War II only to be roundly rejected by Additional Protocol I to the Geneva Conventions and customary international law.  The idea of using civilian pain in order to effectuate political goals would rightly stoke criticisms that the United States’s use of military force against civilian targets equates to acts of sheer terrorism. (See Additional Protocol I art. 51(2) (“Acts or threats of violence the primary purpose of which is to spread terror among the civilian population are prohibited.”) (emphasis added); DOD Law of War Manual, § 5.2.2 (“Measures of intimidation or terrorism against the civilian population are prohibited, including acts or threats of violence, the primary purpose of which is to spread terror among the civilian population.”) (emphasis added).

“We were cautiously optimistic on inflation heading into this year,” as price pressures like those from tariffs were unwinding, said Thomas Ryan, a North America economist at Capital Economics.

“Basically, we’re on hold now, just to see what happens with the energy price shock,” Ryan said. “If it’s long-lasting, we become more concerned about leakage” into other areas of consumers’ wallets, he said.

If you use Signal, you actually have an advantage here, now that you know about this vulnerability. Signal has a setting that blocks the content of messages from appearing in their notifications. That way, even if someone accesses your alerts, all they'll see is you received a Signal message—not who sent it or what it contains.

To turn it on, open Signal, tap your profile in the top-left corner, then hit "Settings." Under Notification Content, choose "No Name or Content" to block all data to the alert. You can compromise here and choose "Name Only" if you want to know who a message is from before you open it—just remember, an intruder may also see you received a message from that person if they scrape your iPhone's notifications.

There has not been a military draft since 1973, when U.S. involvement in the war in Vietnam was winding down.

But in 1980, President Jimmy Carter signed a law requiring men between the ages of 18 to 25 to register for military conscription. Since then, the federal government has relied on voluntary compliance with that law, not automatic enrollment. Men who are 26 are allowed to enroll late to comply with the law.

The agency’s proposal to automatically enroll men in the draft was submitted to the Office of Information and Regulatory Affairs on March 30.

The proposal was made about a month after the U.S. and Israel began a war against Iran.

There are currently no formal plans to reinstate a draft.