abstract / arXiv link

In this paper, we present Accel, a novel semantic video segmentation system that achieves high accuracy at low inference times by combining the predictions of two network branches: (1) a reference branch that extracts high-detail features on a reference keyframe, and warps these features forward using frame-to-frame optical flow estimates, and (2) an update branch that computes features of adjustable quality on the current frame, performing a temporal update at each video frame. The modularity of the update branch, where feature subnetworks of varying layer depth can be inserted (e.g. ResNet-18 to ResNet-101), enables operation over a new, state-of-the-art accuracy-throughput trade-off spectrum. Over this curve, Accel models achieve both higher accuracy and faster inference times than the closest comparable single-frame segmentation networks. In general, Accel significantly outperforms previous work on efficient semantic video segmentation, correcting warping-related error that compounds on datasets with complex dynamics. Accel is end-to-end trainable and highly modular: the reference network, the optical flow network, and the update network can each be selected independently, depending on application requirements, and then jointly fine-tuned. The result is a robust, general system for fast, high-accuracy semantic segmentation on video.

abstract / arXiv link

Models optimized for accuracy on challenging, dense prediction tasks such as semantic segmentation entail significant inference costs, and are prohibitively slow to run on each frame in a video. Since nearby video frames are spatially similar, however, there is substantial opportunity to reuse computation. Existing work has explored basic feature reuse and feature warping based on optical flow, but has encountered limits to the speedup attainable with these techniques. In this paper, we present a new, two part approach to accelerating inference on video. Firstly, we propose a fast feature propagation scheme that utilizes the block motion vector maps present in compressed video to cheaply propagate features from frame to frame. Secondly, we develop a novel feature estimation scheme, termed feature interpolation, that fuses features propagated from enclosing keyframes to render accurate feature estimates, even at sparse keyframe frequencies. We evaluate our system on the Cityscapes and CamVid datasets, comparing to both a frame-by-frame baseline and related work. We find that we are able to substantially accelerate segmentation on video, achieving twice the average inference speed as prior work at any target accuracy level.

abstract / pdf / github

Online and offline storage of digital currency present conflicting risks for a Bitcoin exchange. While bitcoins stored on online devices are continually vulnerable to malware and other network-based attacks, offline reserves are endangered on access, as transferring bitcoins requires the exposure of otherwise encrypted and secured private keys. In particular, fluctuations in customer demand for deposited bitcoin require exchanges to periodically refill online storage systems with bitcoins held offline. This raises the natural question of what upper limit on online reserves minimizes losses due to theft over time. In this paper, we investigate this optimization problem, developing a model that predicts the optimal ceiling on online reserves, given average rates of deposits, withdrawals, and theft. We evaluate our theory with an event driven simulation of the setup, and find that our equation yields a numerical value for the threshold that differs by less than 2% from experimental results. We conclude by considering open questions regarding more complex storage architectures.

abstract / slides

In this work, we propose and evaluate Portal, a payment protocol and software system that enables one-click purchases of long-form news content on the Internet, without requiring a user to sign up for a subscription or login to a content provider's website. The payment protocol enables clients to purchase a digital good (e.g. a single news article) via a standard payment mechanism, such as PayPal or a credit card, and then claim the good from the content provider over an unauthenticated HTTP channel by providing a valid proof-of-payment. This proof demonstrates that 1) a payment transaction of sufficient value was issued for the particular good being claimed (article id verification), and 2) the identity of the payer matches the identity of the client claiming the good (user id verification). Our proposed client-side software handles the construction and provision of this proof, taking the place of manual, password-based authentication.

Our research is motivated by the failure of most news sites to convert a significant percentage of their online readership base to paying subscribers, and various identified shortcomings in the paid subscription model. We present the technical architecture for an alternate monetization system, which allows the purchase of web content on a case-by-case basis, does not lock readers in to a long-standing contract, and does not stipulate that users log in to purchase and read an article discovered while browsing the web. After detailing the payment protocol underling our proposed system, we discuss our particular software implementation, with a focus on key design choices, such as the use of PayPal as a central transaction data store, verified email addresses as universal user identifiers, and client-side certificates for user authentication.

We conclude by evaluating our system on the basis of usability concerns, privacy and security considerations, and questions of adoption and scalability. We then outline key facets of a deployment and launch plan for Portal, ending with a discussion on potential avenues for future work.

abstract / pdf / github

In this paper, we propose and evaluate a software implementation of a Bitcoin micropayments-based revenue system for online businesses, which enables users to make small payments to access web content on a per-use basis, in lieu of viewing ads or signing up for a credit card subscription. We focus in particular on resolving a known issue with past conceptions of micropayment systems, namely that asking users to repeatedly make payment decisions about online content they have not yet experienced imposes a cognitive load, deterring usage. Our solution takes the form of a client-side browser extension, which handles the logistics of making payments, via the use of special HTTP header fields and integration with a client's Bitcoin wallet, but also automates the decision process, by taking appropriate action based on a user's previously indicated preferences. Our system succeeds in eliminating any extra, payment-related actions from the process of browsing the web, a significant step toward removing the mental transactions costs associated with micropayments. We conclude by evaluating our software on the basis of various other criteria, such as ease of installation, security, and scalability, to illustrate avenues for future work in the area.